scripting - creating an iterator in Python from a dictionary in memory-efficient way -

-

I'm running a very large tab-delimited file (containing millions of rows) and depending on it The values ​​of some fields in the file I am pairing lines, eg 

  mydict = defaultdict () for the row in myfile: # Place the group of all rows in the same field in the list in mydict [line.field]. Append (line)   
 Since "MIDDICT" becomes very big, I want to make it in an iterator so I do not have to capture it all in memory instead of making a dictionary I How can I make it, I make an Iterator, by which I can loop and all these lists get the same area value?  
 Thank you.  
 
 "millions of lines" are not too big, unless the lines are long if the lines are long You can save some storage by storing only the files in the file ( .tell ()  /  .seek () ).  
 If the file is sorted by  line.field ; you can use.  
 SQLX <<> Group BY  can help with files of the average size (like using  sqlite ).  
 In fact for larger files you can use.

Comments

Post a Comment

Popular posts from this blog

qt - switch/case statement in C++ with a QString type -

-
I want to use switch-case in my program but the compiler generates an error. How can I use the switch statement with QString ? The compiler gives me this error: The expression of the switch type 'QString' is invalid My code is as follows : boolStopWord (the word QString) {bool flag = false; Switch (word) {case "the": flag = true; break ; "On" case: flag = true; break ; Case "in": flag = true; break ; Case "your": flag = true; break ; Case "near": flag = true; break ; Case "all": flag = true; break ; Case "this": flag = true; break ; } Return flag; } How do I use the switch statement with a Caststring ? You can not use the switch statement in C ++ language with integral or enum types. You can formally enter the square type of object in the switch statement, but this means that the compiler will look for a user-defined conversion to convert it to an integral or enum type.
Read more

python - sqlite3.OperationalError: near "REFERENCES": syntax error - foreign key creating -

-
I'm trying to create a predefined key after checking the manual and I wrote it: Db.execute ("create table" Table 2 (id, integer primary key, somedata integer) "db.execute" (table creation table 2 does not exist (name text, maintain reference (id) ") and found this: sqlite 3. Operation error: near" references ": syntax error Your second ligne is wrong db.execute ("create table if table does not exist 2 (name text, my_id integer, foreign key (my_id) reference Maintenance (ID)) ") as explained
Read more

Python's equivalent for Ruby's define_method? -

-
What is a Python equivalent for Ruby's define_method , which is the dynamic generation of class methods will allow? (As can be seen in Wikipedia) Functions are first-class objects in Python and assign them The properties of a class or an example can be a way of doing the same thing as a Wikipedia example: colors = {"black": "000", "red": "F00", "green": "0f0", "yellow": "ff0", "blue": "00f", "magenta": "fff", "cyan": "0ff", "white": " FFF "} Class MyStream (ARR): Pass for Name, Code in colors.iteritems (): def _in_colour ( W code = code): return '& lt; Span style = "color:% s" & gt;% s & lt; / Span & gt; ' % (Code, self) setter (histrive, "in_" + name, _in_ color)
Read more