Write Clean Python Code Using PipesBy Khuyen Tran, Data Science Intern
Motivation
map and filter are two efficient Python methods to work with iterables. However, the code can look messy if you use both map and filter at the same time.
Image by Author
Wouldn’t be nice if you can use pipes | to apply multiple methods on an iterable like below?
Image by Author
The library Pipe allows you to do exactly that.
What is Pipe?
Pipe is a Python library that enables you to use pipes in Python. A pipe (|) passes the results of one method to another method.
I like Pipe because it makes my code look cleaner when applying multiple methods to a Python iterable. Since Pipe only provides a few methods, it is also very easy to learn Pipe. In this article, I will show you some methods I found the most useful.
To install Pipe, type:
pip install pipe
Where — Filter Elements in an Iterable
Similar to SQL, Pipe’s where method can also be used to filter elements in an iterable.
Image by Author
Select — Apply a Function to an Iterable
The select method is similar to the map method. select applies a method to each element of an iterable.
In the code below, I use select to multiply each element in the list by 2.
Image by Author
Now, you might wonder: Why do we need the methods where and select if they have the same functionalities as map and filter ?
It is because you can insert one method after another method using pipes. As a result, using pipes removes nested parentheses and makes the code more readable.
Image by Author
Unfold Iterables
chain — Chain a Sequence of Iterables
It can be a pain to work with a nested iterable. Luckily, you can use chain to chain a sequence of iterables.
Image by Author
Even though the iterable is less nested after applying chain, we still have a nested list. To deal with a deeply nested list, we can use traverse instead.
traverse — Recursively Unfold Iterables
The traverse method can be used to recursively unfold iterables. Thus, you can use this method to turn a deeply nested list into a flat list.
Image by Author
Let’s integrate this method with the select method to get the values of a dictionary and flatten the list.
Image by Author
Pretty cool!
Group Elements in a List
Sometimes, it might be useful to group elements in a list using a certain function. That could be easily done with the groupby method.
To see how this method works, let’s turn a list of numbers into a dictionary that groups numbers based on whether they are even or odd.
Image by Author
In the code above, we use groupby to group numbers into theEven group and theOdd group. The output after applying this method looks like the below:
[('Even', <itertools._grouper at 0x7fbea8030550>),
('Odd', <itertools._grouper at 0x7fbea80309a0>)]
Next, we use select to turn a list of tuples into a list of dictionaries whose keys are the first elements in the tuples and values are the second elements in the tuples.
[{'Even': [2, 4, 6, 8]}, {'Odd': [1, 3, 5, 7, 9]}]
Cool! To get only the values that are greater than 2, we can add the where method inside the select method:
Image by Author
Note that there are no longer 2 and 1 in the outputs.
dedup — Deduplicate Values Using a Key
The dedup method removes duplicates in a list.
Image by Author
That might not sound interesting since the set method can do the same thing. However, this method is more flexible since it enables you to get unique elements using a key.
For example, you can use this method to get a unique element that is smaller than 5 and another unique element that is larger than or equal to 5.
Image by Author
Now, let’s combine this method with select and where to get the values of a dictionary that has duplicated keys and None values.
Image by Author
In the code above, we:
- remove items with the sameÂ
name - get the values ofÂ
count - only choose the values that are integers.
Within a few lines of code, we can apply multiple methods to an iterable while still keeping the code clean. Pretty cool, isn’t it?
Conclusion
Congratulations! You have just learned how to use pipe to keep your code clean and short. I hope this article will give you the knowledge to turn complicated operations on an iterable into one simple line of code.
Feel free to play and fork the source code of this article here:
Data-science/pipe.ipynb at master · khuyentran1401/Data-science
I like to write about basic data science concepts and play with different algorithms and data science tools. You could connect with me on LinkedIn and Twitter.
Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these:
Python Clean Code: 6 Best Practices to Make your Python Functions more Readable
4 pre-commit Plugins to Automate Code Reviewing and Formatting in Python
Pydash: A Kitchen Sink of Missing Python Utilities
3 Python Tricks to Read, Create, and Run Multiple Files Automatically
Bio: Khuyen Tran is a prolific data science writer, and has written an impressive collection of useful data science topics along with code and articles. Khuyne is currently looking for a machine learning engineer role, a data scientist role, or a developer advocate role in Bay Area after May 2022, so please reach out if you are looking for someone with her set of skills.
Original. Reposted with permission.
Related: