reset password
Author Message
rabbott
Posts: 1649
Posted 09:16 Jan 24, 2019 |

Yesterday, in talking about generating primes, it was suggested (and I agreed) that this list comprehension

[p for (_, p) in zip(range(12), primes())]

was space-inefficient because it stores a pair (a 2-tuple) for each prime.

That's not right. The comprehension generates a 2-tuple for each prime, but it immediately extracts the prime and stores only that. The 2-tuple is discarded once the prime is extracted.

wcwir
Posts: 75
Posted 18:33 Jan 25, 2019 |

Good to know! 

I imagined the zipped list persisted until the for loop was done -- it's good that Python is smarter than that.

rabbott
Posts: 1649
Posted 19:27 Jan 25, 2019 |

Right.  The only thing that's kept is whatever is constructed to the left of the for. All the rest is temporary. 

jpatel77
Posts: 44
Posted 19:39 Jan 25, 2019 |

Yes! , and in fact zip doesn't really create the concrete list at all. All it does is returns a generator function, that spills out the sequential values as it gets called until it gets exhausted. So, for every iteration in for loop, it generates the next logical value in the sequence on demand, and returns that back. If you just put this zip statement inside the list(), it instantly becomes a space-inefficient solution! (I can't stress enough how important this is. This literally constructs the list of tuples in memory and then iterates over it)

If you like, you can go over this doc that explains how zip is implemented. You may find some cool concepts over there, like yield, which basically is responsible for this behavior.

Last edited by jpatel77 at 19:41 Jan 25, 2019.
wcwir
Posts: 75
Posted 19:43 Jan 25, 2019 |

Thanks Jay. I've been using zip to create lists of tuples, so I better check the docs out and see what exactly I've been doing...