Automating Video Recordings in Bash

I didn’t write about my scripts in a long time. This here is a simple script that automates video editing and uploading to a remote host. I’m not sure who’s the right audience for this: if you’re experienced in bash scripts, you won’t find all the explanations needed. If you’re a novice, you might find the script too confusing. I hope you find this interesting, and please feel free to comment, I’m learning more every day.

This script is made especially to automate my journal video recording process. I often record such videos when I brainstorm and need to speak out loud, or sometimes when I want to send a video to a family member. I prefer it to a phone call1. Here is what it does:

  1. Collects all segments of webcam videos (I often record in parts)
  2. Compresses them by a lot - a gig of a video will be about 100MB or less.
  3. Increases the volume by 100% (voice is more important than quality)
  4. Offers the option to upload to a remote server through scp

Ready to jump in? By the time this is posted, I hope to have the full script on GitLab. here it is. I will also include parts here as I explain. OK, here we go.

filename=$(date +%F).mp4

The first segment of the script creates a variable, “filename”, from the current date. It uses the command date (you can read up about it a bit more here or like any command in Linux, use man date to learn more about it) with the +%F option to output a date in a yyyy-mm-dd format. We then add “.mp4” to the end, making it a proper file name for the video we’re going to create.

. $HOME/Personal/scriptsettings

We then source out additional settings from my folder in ~/Personal/scriptsettings. The $HOME is an environment variable that comes “built-in” in Linux and always points to the user’s home folder (also known as “~” as in ~/Documents or ~/Pictures). The dot is the same as typing source as a command. Why source variables from a different location? This will become clear below.

function scp_remote {
    scp $vid_in_path$filename go:$jvid_out_path

Next, we create a function. A function is nothing more than a piece of script that loads into memory to be used later. It will not run unless it is called. The usefulness of the function is that it can be called many times and in different locations throughout the script, an effective way of telling the computer to go back to line so and so and run a line of code. The function here is called “scp_remote”, and all it does is use scp command to connect to a remote server. The variables you see, $vid_in_path and $jvid_out_path are two kinds of variables that are stored in ~/Personal/scriptsettings, which I mentioned above. In that file, $vid_in_path, is defined as $HOME/Videos/Webcam. $vid_in_path is immediately followed by $filename, which we created above. What is actually passed to the computer, if I was to run this on May 29th of 2021, is scp ~/Videos/2021-05-29.mp4.

The next part of the function looks a bit odd. go:$jvid_out_path.

You might know that the scp command needs the local file, which we just covered above, and then after space, the name of the user at a host followed by the destination of where we want to upload the local file to. If you’re following along, you’d probably expect something like scp $vid_in_path$filename JTR@192.168.1.23:/home/jtr/videos. So what’s going on here?

I am using my local ssh config file, which has specific definitions that contain things like the hostname (which is “go” in this case), the IP address, user, and port. When I use ssh (or scp, which is virtually the same for this purpose), I can type scp <local file> go:<remote path>. SCP uses the settings defined in my ssh config. To read more about it, look at articles like this one. This becomes very handy if you have several hosts with different IPs, ports, and usernames. Combined with ssh keys (read more about ssh and ssh keys), you have a quick seamless connection to your remote source which is ideal for scripts like this one. You will also never need to look up which port your ssh connection is configured for (I hope you’re not using the default port!) or expose sensitive IP addresses to the public when writing a blog post like this one.

function upload_remote {
    read -r -p "Upload to remote server? y/n" ans
    case $ans in
      y)
	  scp_remote
	  ;;
      n)
	  exit 1
	  ;;
      *)
	  echo "not a valid answer."
	  upload_remote
    esac

The next part of the script creates another function, upload_remote. This function interacts with the user and asks for input, (y)es or (n)o.

the read command tells the computer to record the input from the keyboard. The -p option creates a prompt, the question. It is similar to typing out echo "Upload to remote server? y/n" followed by read ans. This way, we just use one line of code. the -r switch does not let the user use a special character to get around typing anything that is not already on the keyboard. This last part is just good practice, from what I gather, but it’s not necessary. Finally, the read commands records the answer the user inputs into the variable ans.

The case statement is ideal in cases such as these (that’s why it’s called case: case a, case b, etc…), where you want a basic if-then kind of functionality to interact with a user. Read more here or in similar articles. The basics of what’s going on is this:

  1. read the variable ans which contains the answer from the user.
  2. if ans contains “y” execute the function “scp_remote”
  3. if ans contains “n”, exit, end the script (this is what exit 1 is for).
  4. if ans contains anything else, tell the user it’s not a valid answer and run this function again.

By the way, the above is a good example of why functions are useful: our case statement contains a loop: as long as the user does not give a valid answer (as indicated by case “*”, which stands for “in case you get anything else besides y or n), it will keep asking for a valid one, by summoning itself over and over. It sounds more complicated than it actually is if you look at the code.

for file in ~/Videos/Webcam/*.webm; do
    echo "file '"$file"'" >> $vid_in_path/vids.txt;
done

FFmpeg offers two ways to concat videos. In my experience, concat demuxer, which uses a list specifying the files to concat, works best.

We continue with a loop. The loop creates a variable, “file”, and then tells the script to read each file that ends with “webm” extension ( a free media format that is commonly used for webcam recordings) and echo (print it out) in the following way: file '<name of the file as it is in our variable, also called "file">'. The loop will read each filename and write it into a vids.txt, which I is placed in the path defined in $vid_in_path2.

so for instance, vids.txt may look like this: file ‘2021-05-29-072831.webm’ file ‘2021-05-29-074321.webm’ file ‘2021-05-29-081131.webm’

As you can see from the names of these files, these are segments of a recording that I recorded in parts. I pressed stopped for whatever reason and continued later.

ffmpeg -f concat -safe 0 -i $vid_in_path/vids.txt -vf scale=640:-1 -af volume=2.0 -crf 30 $vid_in_path$filename && rm $vid_in_path/vids.txt

Now we finally get to the ffmpeg part. We use ffmpeg to join (concat) individual videos from the webcam, and then use a strong compression and lower their resolution. The ffmpeg code is a bit condensed, and to be honest, I don’t know exactly what every little thing does, but here’s the general idea:

ffmpeg -f concat -safe 0 -i: this runs ffmpeg with the command to concat. The safe switch is necessary because without it, ffmpeg will complain about the process not being “safe” and quit. the -i is used every time we are manipulating a video.

next, $vid_in_path/vids.txt points to the same list we created above. This variable, vids_in_path, just points to my user’s Video folder, but I wanted to create a variable in case I have video files created elsewhere.

-vf scale=640:-1 is how we tell ffmpeg to reduce the resolution to 640 pixels wide. I don’t need much more of this for my journal videos. -af volume=2.0 asks ffmpeg to increase the volume by 100% (twice as much), so if I speak quietly I can still hear it later. -crf 30 is the compression. There’s a lot more than what I’m writing here (see link), but for now, know that crf 23 is the default, so 30 is higher than default. $vid_in_path$filename gives out the path of the output file. Ffmpeg “knows” to covert the wbem format to mp4 format because of the extension we gave the output file back in the beginning. When this is done and the output file is created (and only after it is created) the vids.txt list is deleted. We don’t need it anymore, the script generates a new one each time.

Finally, we summon the function “upload_remote” which was discussed earlier. After the output video file is complete and saved, the script asks us if to upload to the remote server.

And here we are. That’s all for this time. I hope this was interesting and/or useful. If so, let me know, so I know if I should post similar content.

Thanks for reading!

Footnotes

  1. I hate phone calls with passion. They always interrupt whatever I do, the ring cuts through whatever I may be doing on my phone and demands full attention. 9 times out of 10, there’s nothing urgent that can’t be texted or emailed anyway. ↩︎

  2. The idea of creating a text file to concat videos just to delete it seconds later seems like a waste to me. Since I don’t know of another good way to concat videos on ffmpeg, I was toying around with the idea of creating an array instead and try to pass the array directly into ffmpeg. So far this failed. It seems a bit beyond my current level of scripting, but I’ll get there. ↩︎