Archive for the ‘Linux’ Category

Imagine you have two text files, one with a list of names and another with a list of birth dates, which each name corresponding to the date of birth on the same line number in the other file, like so: names.txt Anthony Kiedis Flea Chad Smith John Frusciante dob.txt 1-Nov-1962 16-Oct-1962 25-Oct-1961 5-Mar-1970 You would […]

Bash gotcha: function variable scope

Posted: 11th October 2015 by Tim in Bash, Linux
Tags: , , , , , , ,

Consider the following bash script: #!/usr/bin/env bash function myFunc {     myvar=123     echo “myFunc: setting myvar=$myvar” } myvar=1 echo “before myFunc: myvar=$myvar” myFunc echo “after myFunc: myvar=$myvar” The code here is fairly simple – we set a variable myvar, call a function and print the value of myvar to the terminal. However, even though we don’t […]

How to get a Makefile directory path

Posted: 27th September 2015 by Tim in Linux, Make
Tags: , , , , , ,

Sometimes in a Makefile you need to reference something from the directory where the Makefile lives. $(PWD) won’t work in this instance, since that will return the path from where make was called which may be different if, for example, you use the –directory option. In this case, we can use the following code to […]

If you’re using a linux terminal and need to convert all upper case characters to lowercase, there are a number of ways you can do it. One of the easiest is to use tr: tr ‘[:upper:]’ ‘[:lower:]’ For example: $ echo “HeLlo WOrlD” | tr ‘[:upper:]’ ‘[:lower:]’ hello world :upper: and :lower: can be reversed […]

Reading a file line by line in Bash

Posted: 26th July 2015 by Tim in Bash, Linux
Tags: , , , , , , ,

There are a few ways to read a file in bash, each with their own caveats. If you’re looking to read a file line-by-line verbatim, including blank lines, then using a simple while loop should do the trick. For example, the following code will print the contents of a file with line numbers: line_no=0 while […]

Pause in Bash

Posted: 27th June 2015 by Tim in Bash, Linux
Tags: , , , , , , ,

Sometimes it is useful to pause a script until the user is ready to proceed. In Windows Batch scripting, this can be done with the PAUSE command. In Bash, the same thing can be done using read with the right parameters: read -rn1 -p “Press any key to continue” This command is doing the following: […]

Awk is a useful language for many command line tasks. Often awk is used on the command line on its own or with strings piped to it, but it is possible to turn that awk code into an executable script. Consider the following script. This file contains awk code with a shebang of awk -f. […]

Linux Terminal Clock

Posted: 9th March 2015 by Tim in Bash, Linux
Tags: , , , , , , , , ,

Ever wanted to have a real-time clock on your linux terminal? We can create one with a single line of bash, like so: while echo -en “$(date)\r”; do sleep 1; done Let’s look at how this works: date is a common unix tool used to print the current date and time. The $(…) means that […]

With Docker, you can specify the command to run inside the container on the command line. But what if you want to run multiple commands? You can’t escape the && syntax, or wrap the command in quotes, as Docker won’t recognise it. The trick here is to use sh -c ‘<command1 && command2 [&& command3 […]

Listing directories only in linux

Posted: 9th February 2015 by Tim in Linux
Tags: , , , , , ,

The ls command will list files and directories in the current directory. But what if you only want to list the directories? There are a few ways to do this. One of the easiest ways is this: ls -d */ The -d flag instructs ls to display directories instead of displaying directory contents. The */ […]

Test if a directory is empty in Bash

Posted: 29th January 2015 by Tim in Bash, Linux
Tags: , , , , , ,

Using Bash, there are a number of ways to test if a directory is empty. One of those ways is to use ls -A to list all files, including those starting with . and see if anything is printed. This can be done like so: if [ ! “$(ls -A <path>)” ] then     echo “<path> […]

There are many ways in the linux terminal to print the nth word of a given file or output. One way to do this without worrying about tabs, extra spaces or word length is to use awk. With awk, this can be done on one line by using the {print $<n>} syntax. For example, the […]

Hide grep command from ps output

Posted: 28th October 2014 by Tim in Linux
Tags: , , , ,

When searching for a running process on a *nix machine, it is common to use ps together with grep, like so: ps -ef | grep vim This works well enough for quick searches, but will always return the grep command since grep vim will always contain the string “vim”. The pgrep utility can work around […]

If you have a JAR file and want to print the details from MANIFEST.MF, this can be done with one command in linux, using the unzip utility. For example: $ unzip -p /usr/share/java/hsqldb.jar META-INF/MANIFEST.MF Manifest-Version: 1.0 Created-By: 1.7.0_03-b147 (Oracle Corporation) Specification-Title: HSQLDB Implementation-Title: Standard runtime Class-Path: /usr/share/java/servlet-api-3.0.jar Main-Class: org.hsqldb.util.SqlTool Ant-Version: Apache Ant 1.8.2 Implementation-Vendor: buildd […]

Command Line Arguments in Bash

Posted: 14th June 2014 by Tim in Bash, Linux
Tags: , , , , ,

In Bash, arguments passed in on the command line are stored in numbered variables. For example, the first argument is $1, the second argument is $2, and so on. The total number of arguments passed to the program is stored in $# $0 contains the path to the program. This path may be an absolute […]

Using tools such as ps or top, you are able to see the processes running on a machine. However, you can’t see the directory from which the process was started. Knowing the working directory can be useful if, for example, you need to move a script or program to stop a fork bomb, if you […]

There are many linux tools available to do search and replace, with sed being one of the most commonly used. However, tools like sed work line-by-line. If you need to replace/remove newline characters then things get complicated. It can be done with sed, but it’s not pretty. The nicest solution I’ve seen is using awk. […]

Perl regular expressions are slightly different from grep (or egrep) regular expressions. grep is sufficient most of the time, but sometimes you may need the extra flexibility of Perl regular expressions, or may just want to test out a regular expression that you will use later in Perl code. This can be done on the […]

Do until script succeeds in bash

Posted: 12th September 2013 by Tim in Bash, Linux
Tags: , , , , , ,

Occasionally you want to continue retrying a script or program until it succeeds. In bash, this can be done using the until … do … done syntax, like so: until (<command>); do echo “FAIL!”; sleep 10; done For example, if you wanted to halt processing until a program.log file was created, you could do something […]

Editing a file in hex mode in Vim

Posted: 27th July 2013 by Tim in Linux, Vim
Tags: , , , , , ,

Native Vim doesn’t have a hex editing mode built in, however it is possible to edit a file in hex by converting the file back and forth using xxd. To do this, open your file in Vim and run :%!xxd. From here you can change the hex values and, when you’re done, run :%!xxd -r […]

There are a few ways to find out if a string contains a substring using bash. Below are a couple of ways this can be done without invoking any other processes. Star Wildcard One very simple method is to match strings using the * character to denote any number of other characters. For example: if […]

If you need to find the UUID of a hard disk in Linux, to add an entry to fstab for example, you can use the blkid command. There are many other methods available, but blkid prints the UUID with the label and disk type, and doesn’t require you to enter the path to the disk. […]

Sed is a useful tool for editing strings on the command line. Changing characters to uppercase or lowercase can be done easily with this tool by simply adding one or more of the following to your substitution string: * \L – convert all proceeding characters to lowercase * \U – convert all proceeding characters to […]

Show line numbers in vi or vim

Posted: 7th January 2013 by Tim in Linux, Vim
Tags: , , , , , , ,

If you want line numbers to show up in vi or vim, this can be done by typing the following command from within the program: :set number or, in shorter form: :set nu Doing this from within vi[m] will show the numbers for that session, but they will be gone next time you start up […]

Say you’ve got a shell script which outputs a bunch of text with some parts colored, such as a regression testing system that colors failures in red and passes in green. The output from this is too big for one screen, so you pipe it through less, which strips away the colors. What you want […]