I’m a network guy, not a programmer. In fact I think my brain can’t operate in an algorithmic way. But in today’s networks (and not just the networks) there is a crucial need for automation.
In my work, I often face repeatable tasks which could be done automatically. But as I said “I’m not a programmer”…And that is a typical response given by every network engineer I have ever talked to during those several years, every time I asked about automation in their workplace. “Do you have to do plenty of repeatable things on many of your nodes or devices?”, “Do you have to manually add or change configurations over and over again?” If the answer is YES, then you’re in good hands :). If the answer is NO, but you are still reading it :), then maybe you will find a few helpful information around here.
When I had noticed how the simple and repetitive tasks could be done automatically, I started to look for an useful tool which could help me to deal with that. At that time, I had a little knowledge of Python, cause I hadn’t had any dealings with the programming since I had my first job as a software tester, where Python was what we mainly used for testing. And so I started to refresh my memory since I hadn’t use Python for a few years. Different networking vendors also noticed the need for automation. In my everyday job I work mainly with the Juniper Networks equipment and there is a dedicated package for Juniper devices called PyEZ. You don’t have to be a programmer to use it :).
This post is the first one in the series about automation in general. Also, during this “journey” I will try to experiment with something more than just the simple scripts used to get my job easier. I’m writing those posts mainly for myself in order to create something like a journal, to which I can go back to anytime and refresh some things whenever I need them. If you find it useful, that’s great. If you find any inconsistency or errors, or if you have any suggestions, feel free to leave a comment.
I’d like to start this series with something I had, and still have a problem(besides actual programming :)). Let’s say that you choose Python as your programming language or scripting tool. You run through the basic tutorial, know the data types, flow control and many other basic stuff. Now, you’re trying to create your own script or your first programme (not the Hello World :)). And then, you face the wall. How to look for a specific function or package that you need for your application? What can you do with a specific data type that you work with? Are those questions familiar to you? For me, that was very difficult and discouraging. So in the part 1, I would like to share with you my ways of using the built in help and how do I search for things I need in my scripts, like classes, modules and functions.
Firstly let’s use the dir function:
C:\Users\user>python
Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:14:34) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']
You will see several objects available from the start. You should be interested in __builtins__ since that module contains all functions and types which are built into Python and are always available.
>>>dir(__builtins__)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException', 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning', 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError', 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FileExistsError', 'FileNotFoundError', 'FloatingPointError', 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError', 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'ModuleNotFoundError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError', 'RecursionError', 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning', 'StopAsyncIteration', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning', 'ValueError', 'Warning', 'WindowsError', 'ZeroDivisionError', '__build_class__', '__debug__', '__doc__', '__import__', '__loader__', '__name__', '__package__', '__spec__', 'abs', 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable', 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit', 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview', 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property', 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars', 'zip']
To learn more about a function or module, use help statement to see the usage of that object:
>>>help(len)
Help on built-in function len in module builtins:
len(obj, /)
Return the number of items in a container.
In the above example we can see what that specific function is doing. As we can see, the len function returns the number of items from some data. For strings it will be number of letters:
>>>name = 'Tomek'
>>>len(name)
5
>>>len('Tomek')
5
In such a way you can check what a specific function or data type is doing. Let’s look at another example. We will create a specific data type like a list and then, we will verify what it is and what we can do with it:
>>>a = [] # Create an empty list named a
>>>type(a) # Verify what type is our new variable
# It is a list
>>>dir(a)
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']
As you can see, the list data type has several functions. Let’s look closer at one of them:
>>>help(a.append)
Help on built-in function append:
append(...) method of builtins.list instance
L.append(object) -> None -- append object to end
And let’s test what we just found out:
>>>a.append('Tomek')
>>>a
['Tomek']
>>>a.append('John')
>>>a
['Tomek', 'John']
Since „a” was an empty list, by issuing the a.append(‘some string’) we have added string type element at the end of our list. Then we can add as many elements as we want and not only the strings:
>>>a.append(10)
>>>a
['Tomek', 'John', 10]
>>>for i in a:
... print(type(i))
...
As you can see, we have added the third element which is a number (integer to be exact).
With a combination of “dir” function and “help” function, you can verify what other functions are available from the start. But what about a functions and other modules that are not in __builtins__ module? We can display the list of other available modules with the help function:
>>>help('modules')
Please wait a moment while I gather a list of all available modules...
__future__ aifc idlelib sched
_ast antigravity idna scp
_asyncio argparse imaplib secrets
_bisect array imghdr select
_blake2 asn1crypto imp selectors
_bootlocale ast importlib serial
_bz2 asynchat inspect setuptools
_cffi_backend asyncio io shelve
...
Enter any module name to get more help. Or, type "modules spam" to search
for modules whose name or summary contain the string "spam".
To verify what object is available in a specific module you must import it first:
>>>import netaddr
>>>dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'a', 'i', 'list', 'name','netaddr']
>>>dir(netaddr)
['AddrConversionError', 'AddrFormatError', 'EUI', 'IAB', 'INET_PTON', 'IPAddress', 'IPGlob', 'IPNetwork', 'IPRange', 'IPSet', 'N', 'NOHOST', 'NotRegisteredError', 'OUI', 'P', 'STATUS', 'SubnetSplitter', 'VERSION', 'Z', 'ZEROFILL', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_sys', 'all_matching_cidrs', 'base85_to_ipv6', 'cidr_abbrev_to_verbose', 'cidr_exclude', 'cidr_merge', 'cidr_to_glob', 'compat', 'contrib', 'core', 'eui', 'eui64_bare', 'eui64_base', 'eui64_cisco', 'eui64_unix', 'eui64_unix_expanded', 'fbsocket', 'glob_to_cidrs', 'glob_to_iprange', 'glob_to_iptuple', 'ip', 'iprange_to_cidrs', 'iprange_to_globs', 'ipv6_compact', 'ipv6_full', 'ipv6_to_base85', 'ipv6_verbose', 'iter_iprange', 'iter_nmap_range', 'iter_unique_ips', 'largest_matching_cidr', 'mac_bare', 'mac_cisco', 'mac_eui48', 'mac_pgsql', 'mac_unix', 'mac_unix_expanded', 'smallest_matching_cidr', 'spanning_cidr', 'strategy', 'valid_eui64', 'valid_glob', 'valid_ipv4', 'valid_ipv6', 'valid_mac', 'valid_nmap_range']
>>>dir(netaddr.IPAddress)
['__add__', '__and__', '__bool__', '__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__hex__', '__iadd__', '__index__', '__init__', '__init_subclass__', '__int__', '__isub__', '__le__', '__long__', '__lshift__', '__lt__', '__module__', '__ne__', '__new__', '__nonzero__', '__oct__', '__or__', '__radd__', '__reduce__', '__reduce_ex__', '__repr__', '__rshift__', '__rsub__', '__setattr__', '__setstate__', '__sizeof__', '__slots__', '__str__', '__sub__', '__subclasshook__', '__xor__', '_module', '_set_value', '_value', 'bin', 'bits', 'format', 'info', 'ipv4', 'ipv6', 'is_hostmask', 'is_ipv4_compat', 'is_ipv4_mapped', 'is_link_local', 'is_loopback', 'is_multicast', 'is_netmask', 'is_private', 'is_reserved', 'is_unicast', 'key', 'netmask_bits', 'packed', 'reverse_dns', 'sort_key', 'value', 'version', 'words']
>>>help(netaddr.IPAddress.version)
Help on property:
the IP protocol version represented by this IP object.
And the example:
>>from netaddr import IPAddress
>>>ip=IPAddress('192.168.1.1')
>>>type(ip)
>>>ip.version
4
In such a way you are able to verify what modules and functions are available for you as the addition to the built-in functions.
Finally, if the “help” and “dir” functions are not enough or you can’t find what you’re looking for, you can use pip to search for specific packages:
C:\Users\user>pip search netaddr netaddr (0.7.19) - A network address manipulation library for Python INSTALLED: 0.7.19 (latest) netaddress (0.2.2) - Validating parser for URI's (RFC 3986)
Of course pip will provide you with much more packages than in this example, since you probably won’t be able to look for the exact name of the package.
If all of the above methods actually fail for you, you can always ask Google. In most cases it will return the stackoverflow links :). Btw. stackoverflow.com is a great place to find specific solutions when you know what you want to achieve, but you are stuck and don’t know how to do it in your code. However, although it’s faster sometimes, may occur less satisfactory :). You should also remember that you’re not always online 🙂
Very good article. Maybe it’s a good time to learn the Python like you 😉