The idea for this post came to my mind during the discussion I had with one of the customer engineers staff, regarding new security features that they want to implement. During the conversation, one of the engineers complained that he spends too much time adding address objects to their security appliances. Basically he receives some document (from his manager/business team), where all of the addresses or other configuration objects (for ipsec tunnels, security policies, NAT tables and so on) are described. Then he has to log in to all security devices and configure those address objects through web GUI or CLI. It takes time…a lot of it. So, I thought that the first post about automation is going to evolve around that particular use case.
Of course if you want to implement some automation to your daily tasks, you have to establish basic rules. For example, the documents or specific inputs that you work with have to be formatted in a strictly defined way. The purpose of this post is to show the power of PyEZ combined with the Juniper Networks devices. PyEZ is a Python library for easy (EZ) interaction with the Juniper devices. For this particular case I created a Python script that has a few basic assumptions:
- The script should convert the provided data to the configuration statements understandable to the particular device/vendor and automatically apply them.
- It should read the data from CSV formatted file.
- The CSV file will have two columns. The first column has the name of the object. The second column contains the ip address related to that object.
- For this example I’m using PyEZ and Juniper Networks SRX configuration schema.
- The address-book object will have global scope – not the zone style.
- There are also several less important assumptions, like how the script should behave when the some exception will rise. The same thing applies to how the script will work when for e.g. the device is not reachable and so on.
Below you can find a full script with the comments in code for each section or statement. Feel free to test it and to modify it if needed. Any comments or feedback both positive or negative are always welcome (if constructive :)). Also note that the script is not production ready. So I suggest you not to use it in production environment without any extensive testing :).
[cc lang=”python” tab_size=”4″ lines=”-1″ theme=”blackboard” line_numbers=”true” width=”auto”]
from jnpr.junos import Device
from jnpr.junos.utils.config import Config
from pprint import pprint
# from getpass import getpass
from jnpr.junos.exception import *
from netaddr import IPAddress
from netaddr.core import * # (1)
”’The purpose of this script is to convert the data from CSV file to the Juniper Networks device
configuration schema. It configures the SRX global address-book. The CSV file has two columns with name
and ip address of an address-book object. Essentially from the Excel file you can configure many devices with
the same new address-book without a need for manual configuration”’
file = ‘C:/Users/tkotyczk/Desktop/python_files/address_book.csv’ # (2)
open_file = open(file)
list_of_elements = []
d = {} # (3)
print(‘This is the data from address_book.csv file’)
for line in open_file:
list_of_elements.append(line.strip())
print(line) # (4)
print()
print(‘This is a list from that file’)
print(list_of_elements)
for index in list_of_elements[1:]:
sublist = index.split(‘;’)
d[sublist[0]] = sublist[1] # (5)
print()
print(‘And this is a dictionary that we will work on’)
pprint(d)
print()
print(‘Those are the commands that will be applied to the device’)
print()
for key, value in d.items():
print(‘set security address-book global address ‘ + key + ‘ ‘ + value + ‘/32’) # (6)
print(‘============================================================’)
true_variable = 1
while true_variable == 1: # (7)
question = input(‘Do you want to continue? (y/n): ‘)
if question == ‘y’:
variable = 0
try:
ip_address = IPAddress(input(‘Please provide the IP address of the remote device: ‘))
except AddrFormatError:
print(‘Incorrect IP address format’)
continue
except ValueError:
print(‘Incorrect IP address format’)
continue # (8)
ip = str(ip_address)
print(‘Connection attempt to: ‘+ip)
port = ’22’
# port = input(‘Input port number: ‘)
remote_device = Device(host=ip, port=port)
while variable < 3: # (9)
try:
remote_device.open()
except ConnectTimeoutError:
print(‘Unable to connect. Check connectivity to the remote device’)
variable += 1 # == variable = variable + 1
continue
except ConnectRefusedError:
print(‘Enable NETCONF on your remote device or check your network interface 🙂 !!!’)
break
except ConnectAuthError:
print(‘Input your username and password:’)
user = input(‘Username: ‘)
password = input(‘Password: ‘)
# password = getpass(prompt = ‘Password: ‘) # (10)
remote_device = Device(host=ip, port=port, user=user, password=password)
try:
remote_device.open()
print(“Connected to:”, remote_device.facts[‘hostname’])
configuration = Config(remote_device)
for key, value in d.items():
configuration.load(‘set security address-book global address ‘ + key + ‘ ‘ + value +
‘/32′, format=’set’) # (11)
print(“Config changes: “)
configuration.pdiff() # (12) print the diff between the current candidate and the active configuration
configuration.commit()
print(‘Configuration was saved’)
true_variable += 1 # == true_variable = true_variable +1
# sys.exit() you can always use sys.exit function.
break
except ConnectAuthError:
print(‘Bad login data:’)
continue
except ConnectRefusedError:
print(‘Enable NETCONF on your remote device or check your network interface 🙂 !!!’)
break
except ValueError:
print(“Incorrect input data!!! Try again”)
continue
except ConnectUnknownHostError:
print(“Incorrect input data!!! Try again”)
continue
except ConnectError:
print(‘Enable NETCONF on your remote device or check your network interface 🙂 !!!’)
break # (13)
true_variable += 1
elif question == ‘n’:
break
else:
continue
print(‘Ending script’)
del list_of_elements
del d
open_file.close()
[/cc]
I’m aware that the script is not perfect. As you can see it’s rather “flat”. Instead of creating a script which executes from top to bottom, I could have created some functions for future use. Also this script works with one single device. What if I had 1000s of such devices? – you can read their ip addresses from another file in the same way as the address objects :). But again, this blog is not the programming course :). The main goal here was to show you the capabilities of PyEZ and how simple it is to create some automation scripts.
Looking at that 100+ lines of code and wondering how this is simple?? 🙂
Check this out:
[cc lang=”python” tab_size=”4″ lines=”-1″ theme=”blackboard” line_numbers=”true” width=”auto”]
remote_device = Device(host=ip, port=port, user=user, password=password)
remote_device.open()
configuration = Config(remote_device)
for key, value in d.items():
configuration.load(‘set security address-book global address ‘ + key + ‘ ‘ + value +’/32′, format=’set’)
configuration.commit()
[/cc]
Any differences? This is the “heart” of that script. Those few lines show how easy and powerful the PyEZ could be when properly used. All of the functions which handle the connection, configuration saving and many more are already there.
“What’s the point in containing all of the other things in the script then?”, you may ask. Well, in large part they are unnecessary , but I wanted to have a script that could handle the most common errors, like no network connection or wrong username and so on.
I’m also pretty sure that I forgot to handle some major errors (like a typo in CSV file which could cause a configuration commit error :)). But in my opinion that’s not the point here. PyEZ combined with Python can be a powerful tool to automating repeatable tasks. It also shows that you don’t have to be a programmer to use python in many other disciplines not always related to the IT world.
Here’s the script in action:
[cc lang=”email” tab_size=”4″ lines=”-1″ theme=”blackboard” line_numbers=”true” width=”auto”]
This is the data from address_book.csv file
Name;IP_address
Serwer_testowy;192.168.1.1
SRV_VIRT1;192.168.1.2
SRV3;192.168.1.3
TOMEK;192.168.1.4
Serwer_jacka;172.55.17.32
Alfa;172.15.32.15
This is a list from that file
[‘Name;IP_address’, ‘Serwer_testowy;192.168.1.1’, ‘SRV_VIRT1;192.168.1.2’, ‘SRV3;192.168.1.3’, ‘TOMEK;192.168.1.4’, ‘Serwer_jacka;172.55.17.32’, ‘Alfa;172.15.32.15’]
And this is a dictionary that we will work on
{‘Alfa’: ‘172.15.32.15’,
‘SRV3’: ‘192.168.1.3’,
‘SRV_VIRT1’: ‘192.168.1.2’,
‘Serwer_jacka’: ‘172.55.17.32’,
‘Serwer_testowy’: ‘192.168.1.1’,
‘TOMEK’: ‘192.168.1.4’}
Those are the commands that will be applied to the device
set security address-book global address Serwer_testowy 192.168.1.1/32
set security address-book global address SRV_VIRT1 192.168.1.2/32
set security address-book global address SRV3 192.168.1.3/32
set security address-book global address TOMEK 192.168.1.4/32
set security address-book global address Serwer_jacka 172.55.17.32/32
set security address-book global address Alfa 172.15.32.15/32
============================================================
Do you want to continue? (y/n): y
Please provide the IP address of the remote device: x.x.x.x
Connection attempt to: x.x.x.x
Input your username and password:
Username: tomek
Password: ———
Connected to: LAB_TESTOWY_SRX
Config changes:
[edit security address-book global]
address google-public-dns { … }
+ address Serwer_testowy 192.168.1.1/32;
+ address SRV3 192.168.1.3/32;
+ address TOMEK 192.168.1.4/32;
+ address Serwer_jacka 172.55.17.32/32;
+ address Alfa 172.15.32.15/32;
Configuration was saved
Ending script
[/cc]
And here is the entire script explained:
- (1) All the imports needed for the scripts. In this place you can use external modules and libraries. Note that getpass module is grayed out. I can use it to hide the password for the device when I’m typing it but it works only in command line, not in the IDE like for eg. Pycharm
- (2) In that variable I’m defining the input file for the script.
- (4) The data from csv file is translated to the python list data type. That is why I’m creating the two variables. An empty list and dictionary (3).
- (5) From list the input data is then converted to the dictionary, because its easier to work with.
- (6) This is a junos command definition but for now its just for presentation purposes.
- (7) This is main loop. The scrip will be executing as long as the true_variable quals 1. I can break it by adding any value to that variable.
- (8) This section is responsible for any typing errors.
- (9) Another while loop. It is used for the device connectivity check. If after 3 atempts the device is nor reachable the script should stop.
- (10) This is a possible getpass example.
- (11) Junos command are being applied to the device with the load function. Note that we have to specify the format.
- (12) Pdiff fuction allows you to see the differences of the configuration after set commands was applied. Its equivalent to show | compare command.
- (13) This entire section is responsible for any error handling.
I hope you liked this post. The next action plan is to create the script that could do the reverse. Currently I’m working on the script which reads the Junipers SRX policy configuration and saves it to the CSV file for reuse in Excel. Stay tuned… 🙂