Computers and storage devices are essential in the digital age, but they can fail or get damaged for various reasons, such as hardware issues, viruses, or human errors. Losing important data can have serious and expensive consequences. Therefore, we need to back up our files regularly and reliably. Manual backups are unreliable, inefficient, and prone to errors. You may skip them, do them incorrectly, or overwrite the wrong files. Also, the backup options in your operating system may not match your preferences or needs. You need a better way to back up your data that is easy, fast, and customizable.

One of my friends recently had a big problem when the hard drive on his desktop crashed. He was not backing up often, so his backups were more than a month old. I felt helpless as I watched him deal with the consequences. I did not want this to happen again, so I set out to find a solution. After a few days of trial and error, I finally came up with a Python script that can automatically, quickly, and reliably back up folders.

This is how I created a Python script that can copy a source folder to a destination folder with a timestamp while skipping certain sub-folders and files we specify. I will describe the steps I took in detail.

Set up the project

Create a new folder in windows

1. press Win+R open run box and type terminal

2. navigate to the folder where you want to create your projectc:\my projects\

3. create a folder mkdir auto_back_up

4. open the newly created project folder in VS Code.

Create a virtual environment

1. Press Ctrl+Shift+P to open the command palette.

2. type Python: create environment and press Enter

3. Follow the prompts to create a new virtual environment.

Write the script in Python

1. Create a new page and name it main.py or by any other name you like. But the file extension must be py

2. In the main.py page, import the required modules:

import os
import shutil
import time

1. Define the source folder and the destination folder. Don’t forget to put the source and destination within single quotes and replace forward slash with back slash

source = 'D:/tempt/docs'   # note single quote and back slash
destination = 'D:/back_up'

1. write code for creating dated folders in destination folder

timestamp = time.strftime('%d-%m-%y _ %H-%M-%S')
new_folder = destination + '/' + timestamp
os.mkdir(new_folder)

1. Write code to exclude folders and files from backing up, if required.

exclusions = ['test_folder', '.txt']

1. now, write code to copy files from source to destination, excluding items in the exclusions list

for root, dirs, files in os.walk(source):
    for exclusion in exclusions:
        if exclusion in dirs:
            dirs.remove(exclusion)
    for file in files:
        if not any(file.endswith(exclusion) for exclusion in exclusions):
            src_file = os.path.join(root, file)
            dst_file = os.path.join(
                new_folder, os.path.relpath(src_file, source))
            dst_dir = os.path.dirname(dst_file)
            if not os.path.exists(dst_dir):
                os.makedirs(dst_dir)
            shutil.copy2(src_file, dst_file)

1. write code to keep the latest required directories and delete the older ones

dirs = sorted(os.listdir(destination), reverse=True)
for dir in dirs[5:]:
    shutil.rmtree(os.path.join(destination, dir))

1. Print backup done message

print("Backup done successfully!")

The finished script will be like this:

import os
import shutil
import time

# Define the source and destination
source = 'D:/tempt/docs'   
destination = 'D:/back_up'

# Define exclusion list
exclusions = ['EPIM', '.txt']

# Create a new directory with date and time stamp
timestamp = time.strftime('%d-%m-%y _ %H-%M-%S')
new_folder = destination + '/' + timestamp
os.mkdir(new_folder)

# Copy files from source to destination, excluding items in the exclusion list
for root, dirs, files in os.walk(source):
    for exclusion in exclusions:
        if exclusion in dirs:
            dirs.remove(exclusion)
    for file in files:
        if not any(file.endswith(exclusion) for exclusion in exclusions):
            src_file = os.path.join(root, file)
            dst_file = os.path.join(
                new_folder, os.path.relpath(src_file, source))
            dst_dir = os.path.dirname(dst_file)
            if not os.path.exists(dst_dir):
                os.makedirs(dst_dir)
            shutil.copy2(src_file, dst_file)

# Keep the latest directories we specify and delete the older ones
dirs = sorted(os.listdir(destination), reverse=True)
for dir in dirs[5:]:
    shutil.rmtree(os.path.join(destination, dir))

# Print backup done message
print("Backup done successfully!")

1. press Ctrl+` to open terminal in VS Code and type pip freeze > requests.txt to generate list of dependencies

Schedule the script to run at startup.

1. Create a batch file specifying the path for python.exe and main.py. Find python exe in your project folder\.venv\scripts and main.py in below venv folder:

"c:\auto_back_up\.venv\Scripts\python.exe"
"c:\auto_back_up\main.py"

1. Schedule in Task Scheduler to run batch file to start on startup

That's it! I will blog next on how to schedule the batch file to run on system startup.