When you want to work with information in a text file, the first step is to read the file into memory. You can read the entire contents of a file, or you can work through the file one line at a time.

5.1.1 Reading an Entire File

Let’s start with a file called pi_digits.txt that contains pi to 30 decimal places with 10 decimals per line:

3.1415926535

8979323846

2643383279

Here is a program that opens this file (which is stored in the same directory as pi_digits.txt), reads it, and prints the contents of the file to the screen:

with open('pi_digits.txt') as file_object:

contents = file_object.read()

print(contents)



The first line of the program is where most of the action is:
- The open() function:
  - No matter what we plan on doing with a file, we first need to open access to it. This function needs one argument, which is the name of the file we want to open.
  - The open() function returns an object representing the file. Python stores this object in file_object, which we use on the next line.
- The keyword with closes the file once access to it is no longer needed. This is why we don’t need the close() function.
The second line of the program uses the read() method to read the entire contents of the file and store it as one long string in contents.
When we print the value of contents in the third line, we get the text file back:

3.1415926535

8979323846

2643383279

# There is a blank line here

The only difference between this output and the original file is the extra blank line at the end of the output. The blank line appears because read() returns an empty string when it reaches the end of the file. This empty string shows up as a blank line. If you want to remove the extra blank line, you can use rstrip() in the print statement:

with open('pi_digits.txt') as file_object:

contents = file_object.read()

print(contents.rstrip())



The rstrip() method removes, or strips, any whitespace characters from the right side of a string. Now the output matches the contents of the original file exactly, with no blank line at the end.

5.1.2 File Paths

When you pass a simple filename like pi_digits.txt to the open() function, Python looks in the directory where the file that’s currently being executed (your .py program file) is stored.
If the file you want to open is not in the same directory as your program file, you need to provide a file path, which tells Python to look in a specific location on your system. You can use the relative file path or the absolute file path. (talk to Mr. Finkel about this, or see the Linux curriculum)

5.1.3 Reading Line by Line

When you are reading a file, you often want to examine each line. You can use a for loop on the file object to examine each line from a file one at a time:

filename = 'pi_digits.txt'

with open(filename) as file_object:

for line in file_object:

print(line)



We store the name of the file that we are reading in the variable filename. This is a common convention when working with files. It is a string telling Python where to find the file. It’s not the actual file.
After we call open(), an object representing the file and its contents is stored in the variable file_object. We are using the with syntax to let Python open and close the file properly.
When we print each line, we find even more blank lines.

3.1415926535

8979323846

2643383279

These blank lines appear because an invisible newline character is at the end of each line in the text file. The print statement adds its own newline each time we call it, so we end up with two newline characters at the end of each line: one from the file and one from the print statement. Using rstrip() on each line in the print statement eliminates these extra blank lines:

filename = 'pi_digits.txt'

with open(filename) as file_object:

for line in file_object:

print(line.rstrip())



Now the output matches the contents of the file once again:

3.1415926535

8979323846

2643383279

5.1.4 Making a List of Lines from a File

When you use with, the file object returned by open() is only available inside the with block that contains it. If you want to retain access to a file’s contents outside the with block, you can store the file’s lines in a list inside the block and then work with that list.
The following example stores the lines of pi_digits.txt in a list inside the block and then prints the lines outside the with block:

filename = 'pi_digits.txt'

with open(filename) as file_object:

lines = file_object.readlines()

for line in lines:

print(line.rstrip())



The readlines() method takes each line from the file and stores it in a list. This list is then stored in lines, which we can continue to work with after the with block ends. We then use a simple loop to print each line from lines. Because each item in lines corresponds to each line in the file, the output matches the contents of the file exactly.

5.1.5 Working with a File’s Contents

After you have read a file into memory, you can do what you want with the data. First, we will build a single string containing all the digits in the file with no whitespace in it:

filename = 'pi_digits.txt'

with open(filename) as file_object:

lines = file_object.readlines()

pi_string = ''

for line in lines:

pi_string += line.rstrip()

print(pi_string)

print(len(pi_string))



We create a variable, pi_string, to hold the digits of pi. We then create a loop that adds each line of digits to pi_string and removes the newline character from each line. We print this string and also show how long the string is:

3.1415926535 8979323846 2643383279

The variable pi_string contains the whitespace that was on the left side of the digits in each line, but we can get rid of that by using strip() instead of rstrip().
- - rstrip() - Removes white space at the end of a string
  - strip() - Removes white space at the beginning and end of a string

3.141592653589793238462643383279

Remember that this is a string and not a number. To use something that you read from a file as a number, you need to convert it to an integer or a float using int() or float().

5.2 Writing to a File

One of the simplest ways to save data is to write it to a file. This way you can examine output after a program finishes running. You can also write programs that read the text back into memory and work with it again later.

5.2.1 Writing to an Empty File

To write text to a file, you need to call open() with a second argument telling Python that you want to write to the file. Here, we write a simple message and store it in a file instead of printing it to the screen:

filename = ‘programming.txt’

with open(filename, ‘w’) as file_object:

file_object.write(“I love programming”)

The call to open() in this example has two arguments:
- First argument - The name of the file we want to open
- Second argument - ‘w’ tells Python that we want to open the file in write mode. You can open a file in:
  - read mode ‘r’
  - write mode ‘w’
  - append mode ‘a’
  - read and write mode ‘r+’
- If there is no argument, Python opens the file in read-only mode
After we run the code above, there is no terminal output, but if you open the file programming.txt, you will see one line:

I love programming

Python can only write strings to a text file. If you want to store numerical data in a text file, you will have to convert the data to string format first using the str() function. 

5.2.2 Writing Multiple Lines

The write() function doesn’t add any newlines to the text you write. So if you write more than one line without including newline characters, your file may not look the way you want it to:

filename = ‘programming.txt’

with open(filename, ‘w’) as file_object:

file_object.write(“I love programming.”)

file_object.write(“I love it so much.”)

FILE CONTENTS:

I love programming.I love it so much.

We could include newlines in your write() statement to make each string appear on its own line:

filename = ‘programming.txt’

with open(filename, ‘w’) as file_object:

file_object.write(“I love programming.\n”)

file_object.write(“I love it so much.\n”)

FILE CONTENTS:

I love programming.

I love it so much.

You can also use spaces, tab characters, and blank lines to format your output, as usual.

5.2.3 Appending to a File

If you want to add content to a file instead of writing over existing content, you can open the file in append mode. Any lines you write to the file will be added at the end of the file. If the file doesn’t exist yet, Python will create an empty file for you.
Continuing the example above, we can add more to our love of programming file called programming.txt by writing this program. We use the ‘a’ argument to open the file for appending instead of writing over the existing file.

filename = ‘programming.txt’

with open(filename, ‘a’) as file_object:

file_object.write(“I also love going to school.\n”)

file_object.write(“I plan on passing all of my classes.\n”)

FILE CONTENTS:

I love programming.

I love it so much.

I also love going to school.

I plan on passing all of my classes.

5.3 Exceptions (try / except)

Python uses objects called exceptions to manage errors during a program's execution. Whenever an error occurs that makes Python unsure of what to do next, it creates an object exception.
If you don’t handle exceptions, the program will stop and show a traceback, which includes a report of the exception that was raised.

Example:

print(5/0) # You can’t divide something by zero, so we get an error

ERROR:

Traceback (most recent call last):

File “division.py”, line 1, in <module>

ZeroDivisionError: division by zero

The error ZeroDivisionError is an exception object. We can tell Python what to do when this happens

5.3.1 try / except

The try and except block in Python is used to catch and handle exceptions without stopping the program. Python executes code following the try statement as a “normal” part of the program. The code that follows the except statement is the program’s response to any exceptions in the preceding try clause. The except clause determines how your program responds to exceptions.
Basic structure:

Image provided by RealPython

Here is what a try-except block for handling the ZeroDivisionError exception looks liike:

try:

print(5/0)

except: ZeroDivisionError:

print(“You can’t divide by zero!”)

We put the line that caused the error inside a try block. If the code in a try block works, Python skips over the except block. If the code in the try block causes an error, Python looks for an except block whose error matches the one that was raised and runs the code in that block.
In the above example, we get a more friendly error message instead of a traceback:

You can’t divide by zero!

If more code followed the try-except block, the program would continue running. We will see an example of that in the next section.

5.3.2 Using Exceptions to Prevent Crashes

When asking for user input, we want our program to respond to invalid input appropriately by prompting for more valid input instead of crashing. Here is an example of a simple calculator that does only division:

print("Give me two numbers, and I'll divide them.")

print("Enter 'q' to quit.")

while True:

first_number = input("\nFirst number: ")

if first_number == 'q':

break

second_number = input("Second number: ")

if second_number == 'q':

break

answer = int(first_number) / int(second_number)

print(answer)



This program does nothing to handle errors, so asking it to divide by zero causes it to crash:

OUTPUT:

Give me two numbers, and I’ll divide them.

Enter ‘q’ to quit.

First number: 5

Second number: 0

Traceback (most recent call last):

File “division.py”, line 9, in <module>

answer = int(first_number) / int(second_number)

ZeroDivisionError: division by zero

Having the program crash is not great, but it’s also not good to let users see tracebacks. Nontechnical users will get confused and malicious users will learn more than you want them to see.

5.3.3 The “else” Block

We can make this a better performing program by wrapping the line that might produce errors in a try-except block. The error occurs on the line that performs the division, so that’s where we’ll put the try-except block.
Any code that depends on the try block executing successfully goes in the else block.

print("Give me two numbers, and I'll divide them.")

print("Enter 'q' to quit.")

while True:

first_number = input("\nFirst number: ")

if first_number == 'q':

break

second_number = input("Second number: ")

try:

answer = int(first_number) / int(second_number)

except ZeroDivisionError:

print("You can't divide by 0!")

else:

print(answer)



If the try statement doesn’t succeed because of a division by zero error, we print a friendly message telling the user how to avoid this error. The program continues to run and the user never sees a traceback error.

OUTPUT:

Give me two numbers, and I’ll divide them.

Enter ‘q’ to quit.

First number: 5

Second number: 0

You can’t divide by 0!

First number: 5

Second number: 2

2.5

First number: q

5.3.4 Handling the FileNotFoundError Exception

A common issue when working with files is handling missing files. The file you’re looking for might be in a different location, the filename may be misspelled, or the file may not exist at all. You can handle all of these with a try-except block.
Let’s try to read a file that doesn’t exist. The following program tries to read the content of Alice in Wonderland, but the file alice.txt isn’t in the same directory as alice.py:

Example: alice.py

filename = ‘alice.txt’

with open(filename, encoding = ‘utf-8’) as f_obj:

contents = f_obj.read()



The encoding argument is needed when your system’s default encoding doesn’t match the file that’s being read. (clarify this)
Python can’t read from a missing file, so it raises an exception:

Traceback (most recent call last):

File “alice.py”, line 3, in <module>

with open(filename, encoding=’utf-8’) as f_obj:

FileNotFoundError: [Errno 2] No such file or directory: ‘alice.txt’

The last line reports a FileNotFoundError:. This is the exception Python creates when it can’t find the file it’s trying to open. In this example, the open() function produces the error, so to handle it, the try block will begin just before the line that contains open();

filename = ‘alice.txt’

try:

with open(filename, encoding = ‘utf-8’) as f_obj:

contents = f_obj.read()

except FileNotFoundError:

msg = "Sorry, the file " + filename + " does not exist."

print(msg)

Because the code in the try block produces a FileNotFoundError, Python looks for an except block that matches that error. Python then runs the code in that block with the following message instead of a traceback:

Sorry, the file alice.txt does not exist.

5.3.5 Analyzing Text

Project Gutenberg (gutenberg.org) has entire books as simple text files and they are free to anyone. We are going to pull the text of Alice in Wonderland and try to count the number of words in the text.
We can use the string method split(), which can build a list of words from a string. Here is what split() does with a string containing just the title “Alice in Wonderland”:

Code:

title = “Alice in Wonderland”

print(title.split())

OUTPUT:

[‘Alice’, ‘in’, ‘Wonderland’]

We can use split() on the entire text. Then we will count the items in the list to get a rough idea of the number of words in the text:

filename = ‘alice.txt’

try:

with open(filename, encoding = ‘utf-8’) as f_obj:

contents = f_obj.read()

except FileNotFoundError:

msg = "Sorry, the file " + filename + " does not exist."

print(msg)

else:

# Count the approximate number of words in the file

words = contents.split()

num_words = len(words)

print("The file " + filename + " has about " +

str(num_words) + " words.")



Assuming the file alice.txt is in the correct directory now, the try block will work this time and the else block will be executed. In the else block:
- words is a list consisting of all the words in the book
- num_words becomes the length of the list words, which is our word count.
- We then print the word count

OUTPUT:

The file alice.txt has about 29461 words.

There is some extra information in the file so the count is a little high, but it’s close.

5.3.6 Working with Multiple Files

Let’s add more books to analyse, but first let’s move the bulk of the program to a function called count_words(). It will be easier to work with more books after that:

def count_words(filename):

"""Count the approximate number of words in a file."""

try:

with open(filename, encoding = ‘utf-8’) as f_obj:

contents = f_obj.read()

except FileNotFoundError:

msg = "Sorry, the file " + filename + " does not exist."

print(msg)

else:

# Count the approximate number of words in the file

words = contents.split()

num_words = len(words)

print("The file " + filename + " has about " + str(num_words)

+ " words.")

filename = 'alice.txt'

count_words(filename)



Most of the code is unchanged. (What is a docstring??) Now we can write a loop that calls this function to act on a list of files that we want to analyze.
We can replace the last two lines of the program above to count the number of words in the following texts: Alice in Wonderland, Siddhartha, Moby Dick, and Little Women. We are leaving out siddartha.txt from the directory containing this program so that we can see how the program handles a missing file:

def count_words(filename):

"""Count the approximate number of words in a file."""

try:

with open(filename, encoding = ‘utf-8’) as f_obj:

contents = f_obj.read()

except FileNotFoundError:

msg = "Sorry, the file " + filename + " does not exist."

print(msg)

else:

# Count the approximate number of words in the file

words = contents.split()

num_words = len(words)

print("The file " + filename + " has about " + str(num_words)

+ " words.")

filenames = ['alice.txt','siddhartha.txt','moby_dick.txt','little_women.txt']

for filename in filenames:

count_words(filename)



OUTPUT:

The file alice.txt has about 29461 words.

Sorry, the file siddhartha.txt does not exist.

The file moby_dick.txt has about 215136 words.

The file little_women.txt has about 189079 words.

The missing siddhartha.txt has no effect on the rest of the program’s execution.

Failing Silently:

In the last example, we reported to our users that one of the files was unavailable, but we don’t need to report every exception you catch. Sometime we want to just skip an exception if it occurs and continue as if nothing happened.
To make a program fail silently, you just use pass in the except block. If we wrote pass instead of except FileNotFoundError

except FileNotFoundError:

pass

instead of:

except FileNotFoundError:

msg = "Sorry, the file " + filename + " does not exist."

print(msg)

we would end up with the following output:

The file alice.txt has about 29461 words.

The file moby_dick.txt has about 215136 words.

The file little_women.txt has about 189079 words.

(Maybe add a section on JSON for storing data)

Chapter 5 Assignments

A5.1 pi to 1,000,000 Digits

If we start with a text file that contains pi to 1,000,000 decimal places, we can create a string containing all of these digits. We don’t need to change our program from reading pi with 30 digits except to pass it a different file.
Print the first 50 decimal places of pi and the total number of digits in the string that you build
Use THIS FILE

OUTPUT:

3.141159265358979323846264338327950288419716939937510…

1000002

A5.2 Is Your Birthday Contained in PI?

Expand on your program from A5.1 to find out if your birthday appears anywhere in the first million digits of pi. Express your birthday as a string of digits and search for that string in the file with pi in it.
Hint: use the following if statement:

if birthday in pi_string: # birthday in the form mmddyy

# pi_string holds the million digits of pi

# automatically checks if its in there

Extra challenge: At what digit in pi does the birthday start?

SAMPLE RUN:

Enter your birthdate, in the form mmddyy: 120372

Your birthday appears in the first million digits of pi!!

A5.3 Replacing Words

Write a text file called learning.txt with the following:

I am a student in Finkel’s class. Finkel teaches a lot of good stuff. Whenever I go

to Finkel’s class, I feel like I am expanding my mind, but I’m not sure Finkel would

agree.

Read the file into a Python program and replace the name Finkel with another teacher’s name. Print out the entire file and make sure it worked.

A5.4 Take a Poll

Write a program that asks people why they like programming. Each time someone enters a reason, add their reason to a file that stores all of the responses. Print it out after adding a reason to make sure it is working properly.

A5.5 Checking input

Create a program that asks a user for a number and returns two times that number. The program should include a try/except block that will tell the user that their number is not valid if they enter a string. If a string is entered, the program should terminate without throwing an error.

A5.6 Addition

One common problem when prompting for numerical input occurs when people provide text instead of numbers. When you try to convert the input to an int, you will get a ValueError.
Write a program that prompts for two numbers. Add them together and print the result. Catch the ValueError if either input value is not a number, and print a friendly error message. The program should work in a loop to keep prompting for two numbers until the user presses ‘q’ to quit.

A5.7 Common Words

Using Project Gutenberg (gutenberg.org) find two texts you would like to analyze. Download the text files for those works, or copy the raw text from your browser into a text file on your computer.
Use the count() method to find out how many times the word ‘the’ appears in the text. Hint:

line = “Row, row, row your boat”

print(line.count(‘row’)) # prints 2

print(line.lower().count(‘row’)) # prints 3

lower() makes all of the words lower case.

Reading & Writing Files

5. Reading and Writing Files

5.1 Reading From a File

5.1.1 Reading an Entire File

5.1.2 File Paths

5.1.3 Reading Line by Line

5.1.4 Making a List of Lines from a File

5.1.5 Working with a File’s Contents

5.2 Writing to a File

5.2.1 Writing to an Empty File

5.2.2 Writing Multiple Lines

5.2.3 Appending to a File

5.3 Exceptions (try / except)

5.3.1 try / except

5.3.2 Using Exceptions to Prevent Crashes

5.3.3 The “else” Block

5.3.4 Handling the FileNotFoundError Exception

5.3.5 Analyzing Text

5.3.6 Working with Multiple Files

Chapter 5 Assignments

A5.1 pi to 1,000,000 Digits

A5.2 Is Your Birthday Contained in PI?

A5.3 Replacing Words

A5.4 Take a Poll

A5.5 Checking input

A5.6 Addition

A5.7 Common Words