Python for DevOps: Chapter2 - File Handling and Operations

In the DevOps world, you are parsing, reading, writing, or editing files and Python gives the ability to do all of these. To find the root cause, you read log files, and it comes very handily when you are going with Infrastructure as Code. Automating these file operations is what makes the modern DevOps effective over the traditional system administrators. Rather than manually following the standard operating procedure, you can automate them, this way you can avoid missing any steps and also make sure that the steps are executing in chronicle order.

Reading and Writing File

By using open function, reading and writing to a file is very easy. The open function expects two arguments, one is the file path and the other one is mode (if not defined, it is read).

open_file = open("readme.txt")
text = open_file.read()
print(text)

In the above code snippet, by using the open function created a file object and then using the read method got the content of the file.

It is recommended to close the file when you finish working with it, Python closes the file when it's out of scope but till then it may prevent other processes to access it. To close the file, use the object of the file with the method close.

open_file.close()

You can also use the readlines method to read files, this method splits the file content by new line and returns them in a list of strings.

open_file = open("readme.txt")
text = open_file.readlines()
print(text)
open_file.close()

Here we can iterate through the content of the file by line by line. In the below code snippet, iterating through the content of the file line by line.

open_file = open("readme.txt")
text = open_file.readlines()
i = 1
for line in text:
    print("[" + str(i) + "] " + line)
    i = i + 1
open_file.close()

To write to a file, use write mode (represented as w).

open_file = open("readme.txt", "w")
open_file.write("Some random text")
open_file.close()

The write method will overwrite any existing text. The open function creates a file if it does not already exist and overwrites if it does. If you want to keep existing content and only want to append new texts, use append mode a.

open_file = open("readme.txt", "a")
open_file.write("Appending the text")
open_file.close()

Using file object's read and write functions are good for unstructured text files, however when it comes to JSON``,CSV`` or YML files which are widely used. You can read these files but to use them would be a bit difficult, especially to deserialize them. There are modules available to deserialize these files into data structure and types to the relevant ones.

JSON Files

Python has a built-in package for JSON, called json. To import json module, we use import statement. Let us try to understand the below code.

import json
open_file = open("cities.json","r")
cities = json.load(open_file)
for city in cities:
    print(city["name"])

Here we are using the object of the file to parse the JSON file in an appropriate data format. Here the cities object is of type list of dictionaries because of the JSON file formate.

In a similar way, we can convert Python objects to JSON objects using dumps method. Below is a code example for it.

import json
user_profile = {
    "name" : "Pawan",
    "city" : "Mumbai",
    "status" : "active"
}
data = json.dumps(user_profile)
print(data)

Here the user_profile object is converted to JSON using dumps method.

Using method dump, you can write the content to a file.

YML Files

YAML or YML is another commonly used file type, it is very similar to JSON but it's very compact and uses white spaces to format (like how Python uses whitespaces). The most commonly used library for parsing YAML files in Python is PyYAML. It is not in the Python Standard Library, but you can install it using pip:

pip install PyYAML Once the library is installed, it can be imported and can be used in a similar way as of JSON module. Code example of it:

import yaml
opened_file = open("webserver.yml","r")
webserver = yaml.safe_load(opened_file)
print(webserver[0]["vars"])

The output of it.

There is support for CSV, XML, and some other file formats, to use them is as easy as JSON and YML files as explained above.

That is it in this blog, Keep Learning Keep Shining.