When I first started with Python, I was coming from a C background. I had no extensive software development experience back then, and the taste of freedom provided by dynamic typing was so sweet. Functions leveraging polymorphism and duck typing allowed me to do a lot with a little.
Later, as my experience grew and I became involved in large scale projects, it dawned on me that this freedom is a blessing and a curse. As contributors grow and the code is being pushed closer to production-grade, not having static typing or type checking can lead to nasty surprises.
This was a feeling shared between many in the Python ecosystem. However, keeping the freedom allowed by dynamic typing, but mitigating its negative impact is difficult.
Enter type annotations.
If there is one feature in Python which maximizes the positive impact on your code while requiring minimal effort, it is type annotation. It allows the developer to effectively communicate expected argument types and return values with the interpreter (and other developers as well) while keeping the advantages of dynamic typing.
Dynamic typing in Python
So, what is dynamic typing anyway? To see this, let’s play around a little.
Behind the scenes, variables in Python like
a above are pointers, pointing to objects with a certain type. However, the pointers are not restricted to represent objects of fixed type for a given name. This gives us a lot of freedom. For instance, functions can accept any type as an argument, because in the background, a pointer is passed.
Unfortunately, without proper care, this can go wrong really fast.
Dynamic typing is a double-edged sword
Just think of the following example.
This simple statistical function works for lists and NumPy arrays as well! How awesome!
Well, not entirely. There are plenty of ways to burn yourself with this. For instance, you can unintentionally pass something which causes the function to crash. This is what happens if we call the function with a string.
The division operator is not defined between strings and integers, thus the error. Since Python is an interpreted language, this problem would not surface until the function is actually called with a bad argument. This can be after weeks of runtime. Languages like C can catch these errors in compile-time before anything goes wrong.
Things can get even worse. For instance, let’s call the function with a dictionary.
The execution is successful, but the result is wrong. When a dictionary is passed, the
max functions compute the minimum and maximum of the keys, not the values like we want. This kind of bug can remain undetected for a long time, meanwhile, you are under the impression that things are alright.
Let’s see what can we do to avoid problems like these!
Enter function annotations and type hints
In 2006, PEP 3107 introduced function annotations, which were extended in PEP 484 with type hints. (PEP is short for Python Enhancement Proposal, which is Python’s way of suggesting and discussing new language features.)
Function annotation is simply “a syntax for adding arbitrary metadata annotations to Python functions”, as PEP 3107 states. How does it look in practice?
Types can be hinted with
argument: type = default_value and return values with
def function(...) -> type.
These are not enforced at all and ignored by the interpreter. However, this does not mean that they are not mind-blowingly useful! To convince you, let’s see what can we gain!
Faster development with code completion
Have you ever tried to develop in a barebones text editor like Notepad? You have to type in everything and keep in mind what is what all the time. Even an IDE cannot help you if it has no idea about the object you are using.
Take a look at the example below.
With function annotation, the IDE is aware of the type of the
data object, which is the return value of the
preprocess_data function. Thus, you get autocompletion, which saves a tremendous amount of time.
This also helps when using the function. Most often, the definition is in an entirely different module, far away from where you are calling it. By telling the IDE the type of arguments, it will be able to help you pass the arguments in the correct format without having to manually check the documentation or the implementation.
Code as documentation
Developers spend much more time reading code than writing it. I firmly believe that great code is self-documenting. With proper structuring and variable naming, comments are rarely needed. Function annotation contributes significantly to this. Just a glance at the definition will reveal a lot on how to use it.
The annotations are accessible from the outside of the function.
This is not only useful for the programmer but the program itself as well! Before calling the function, you can check the validity of its arguments at runtime, if needed.
Based on PEP 484, type checking was taken to the next level. It included the
typing module, “providing a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and (perhaps, in some contexts) code generation utilizing type information”, as stated in the PEP.
To give a more concrete example, the
typing module contains
List, so by using the annotation
List[int], you can tell that the function expects (or returns) a list of integers.
One step further: data validation with pydantic
Type checking opens up a lot of opportunities. However, doing it manually all the time is not so convenient.
If you want a stable solution, you should try pydantic, a data validation library. Using its
BaseModel class, you can validate data at runtime.
You can go even beyond this example, for instance by providing custom validators for pydantic models.
pydantic is one of the pillars of the FastAPI, which is the rising star of web development frameworks in Python. There, pydantic makes available to easily define the JSON schemas for the endpoints.
So, I hope that I have convinced you by now. Type annotations require minimal effort, but they have a huge positive impact on your code. It
- makes the code easier to read for you and your team,
- encourages you to have types in mind,
- helps to identify type-related issues,
- enables proper type checking.
If you are not using it already, you should start doing it now! This is one of the biggest code improvements you can do with only a small amount of work.