CIENCE

If you ask any programming wizard the secret to skilled programming, I can assure he will respond with this “ I am not a great programmer; I am just a good programmer with great habits.” It means writing clean code with minimum complexity. It applies to any software programs regardless of what technology the program is developed.Machine learning and data science involve applying some most complex logic.  For some programmers, it is still a puzzle to envisage simple coding structure and machine learning on the same page. However, with best practice, the code complexity in machine learning and data science projects can be minimized.  Ways to reduce code complexity in machine learning and data science projects1)  Implement clean code practiceThere is a well-known phrase among programmers “we will read more code than we will ever write”. Cluttered code makes it difficult to understand and modify. One common issue is keeping “dead code”  in the codebase. Removing dead code gives a clear visual of your code. Try to avoid print statements (even glorified print statements such as head(),df.describe(), df.plot() should be avoided).2)  Variable namesIn software programming, it is a bad practice to embed information about variable types in the variable name. One common problem in data science code is data frames (df). Every data frame is named as df. The good programming practice says, don’t force the reader of your code to translate what the variable means. Instead, try to use a naming convention that is easy to understand. See the example below, with Pandas Python library, the code is written in much human-readable form ( Code 2), and it gives a better idea about what the code is about compared to code 1. ( image source: towardsdatascience)Code (the same above code) declared with a name instead of “df”.  If you compare the above code with the below code, you can quickly figure out which one is easy to understand. ( image source: towardsdatascience)3)  Use functions to simplify code Function simplifies the code usage.  If ‘function” is declared wisely, then programmers don’t have to read each line or search the internet for things that they don’t understand.  Also, to repeat the same transformation on any column, a single line of code would do the job instead of writing lines and lines of code. 4)  Take code out of Jupyter notebooksJupyter notebooks are great for quick prototyping. This is where developer — glue code, , glorified print statements (df.describe() or df.plot()), print statements, unused import statements and even stack traces. But it can make things more complicated as everything gets accumulated in one place. So, whenever a single line of code is changed, the entire notebook has to re-run. It means the longer the notebooks become, the harder it is to get feedback on whether our changes are working. The alternate way is to extract code into functions and Python modules, and have unit tests. With unit tests, the test runner can give feedback on the changes in a matter of seconds, even when there are hundreds of functions.5)  Adopt a test-driven development approachUnlike traditional software development, in test-driven development, you write a test for your code first, make sure it fails, and then create enough code to pass the test.  It means you are only focussed on passing the actual test and not writing code that is intended to pass all possible tests in the future. Once the developer has the code that passes the test, they can clean up the code, remove any duplication, apply good coding style rules, and add documentation. The TDD approach yields short cycles of fail-pass iterations. In the above points, we have mentioned the importance of functions, and the TDD cycle enables the programmer to write functions in small modules.6)  Make small and frequent commitsThe whole development process can greatly benefit from small and frequent commits. Often large changes in code can be difficult to review and takes a significant amount of energy. With small commits, the reviewer can detect early whenever a developer is going in the wrong direction. It also helps to reduce visual distractions and cognitive load.