java - Accessing a mapper's counter from a reducer -

-

I need to access the counters from my Maper in my reducers. Is this possible? If so, how is it done?

As an example: My Mapper is: 

  Public square counter mapper and text text, text, text, text & gt; {Static enum TestCounters {TEST} @ Override throws the protected Zero map (text key, text value, reference reference) IOException, interrupted; Exception {context.getCounter (TestCounters.TEST). Construction (1); Context.write (key, value); }}   
 is my reducer  
  extends the public square CounterReducer Reducer & lt; Text, Text, Text, Longway & gt; {@Override Reduces Safe Zero (text key, iterable & lt; text & gt; values, reference references) throws IOException, interrupted; Exception {counter counter = context.getCounter (CounterMapper.TestCounters.TEST); Long counter value = counter .getValue (); Context.write (key, new long countervalue); }}   
 counterValue is always 0. Am I doing something wrong or is it not possible?   
 
 
 In Reducer's Configurable (JobConf), you see the reducer's own Job ID for the JobConf object With this, your rediscover can create your own job client for job tracker - that is, ask counters for this job (or any work for that matter).  
  // Reduce class ... Private Long Mapper Cropper; @ Override Public Waste Configured (JobConf Config) {JobClient Client = New JobClient (conf); RunningJob parentJob = client.getJob (JobID.forName (conf.get ("mapred.job.id"))); MapperCounter = parentJob.getCounters (). GetCounter (MAP_COUNTER_NAME); }   
 You can now use the mapper control within the least () method.  
 You really need a try-catch. I am using the old API, but should not be friendly to the new API.  
 Note that before the start of any reducer the mappers counter should be finalized, so unlike Justin Thomas's comment, I believe that you should get the exact price (unless the reducer Not equal counter counters!)

Comments

Post a Comment

Popular posts from this blog

qt - switch/case statement in C++ with a QString type -

-
I want to use switch-case in my program but the compiler generates an error. How can I use the switch statement with QString ? The compiler gives me this error: The expression of the switch type 'QString' is invalid My code is as follows : boolStopWord (the word QString) {bool flag = false; Switch (word) {case "the": flag = true; break ; "On" case: flag = true; break ; Case "in": flag = true; break ; Case "your": flag = true; break ; Case "near": flag = true; break ; Case "all": flag = true; break ; Case "this": flag = true; break ; } Return flag; } How do I use the switch statement with a Caststring ? You can not use the switch statement in C ++ language with integral or enum types. You can formally enter the square type of object in the switch statement, but this means that the compiler will look for a user-defined conversion to convert it to an integral or enum type.
Read more

python - sqlite3.OperationalError: near "REFERENCES": syntax error - foreign key creating -

-
I'm trying to create a predefined key after checking the manual and I wrote it: Db.execute ("create table" Table 2 (id, integer primary key, somedata integer) "db.execute" (table creation table 2 does not exist (name text, maintain reference (id) ") and found this: sqlite 3. Operation error: near" references ": syntax error Your second ligne is wrong db.execute ("create table if table does not exist 2 (name text, my_id integer, foreign key (my_id) reference Maintenance (ID)) ") as explained
Read more

Python's equivalent for Ruby's define_method? -

-
What is a Python equivalent for Ruby's define_method , which is the dynamic generation of class methods will allow? (As can be seen in Wikipedia) Functions are first-class objects in Python and assign them The properties of a class or an example can be a way of doing the same thing as a Wikipedia example: colors = {"black": "000", "red": "F00", "green": "0f0", "yellow": "ff0", "blue": "00f", "magenta": "fff", "cyan": "0ff", "white": " FFF "} Class MyStream (ARR): Pass for Name, Code in colors.iteritems (): def _in_colour ( W code = code): return '& lt; Span style = "color:% s" & gt;% s & lt; / Span & gt; ' % (Code, self) setter (histrive, "in_" + name, _in_ color)
Read more