Laboratory 3 - Linux file structure
Third article presenting the Linux file structure, by using basic commands for file and directory operations, regular expressions, pipes, redirection and archives.
Lab 3 - Linux file structure
In this laboratory you will exercise with basic Linux commands, the Bash, File System, FHS, files and directories, pipes, redirection, regular expressions and archives.
Contents:
-
Lab 3.1 Working with commands, searching, quoting, pipes.
- Lab objective 1: Working with simple commands
- Lab objective 2: Change the character encoding
- Lab objective 3: Use search patterns
- Lab objective 4: File globbing
- Lab objective 5: Quoting commands
- Lab objective 6: Working with pipes and redirection
-
Lab 3.2 Working with the file system, directory content and file operations.
- Lab objective 7: Calculate the overall size of each of your system's top-level directories
- Lab objective 8: Take a tour of your filesystem and give extra attention to /proc filesystem
- Lab objective 9: Explore the Linux File system Hierarchy using an installation DVD
- Lab objective 10: Change directories and list directory contents
- Lab objective 11: Create and view files
- Lab objective 12: Perform multiple file operations
- Lab objective 13: Find files in Linux
- Lab objective 14: Search file content
- Lab objective 15: Using the tar utility
- Lab objective 16: Using tar for backup
-
Lab 3.3 Working with grep awk and sed.
- Lab objective 17: Working with grep command
- Lab objective 18: Working with awk command
- Lab objective 19: Working with sed command
-
Lab 3.4 Working with files - more real life scenarios
- Lab objective 20: Working with very large text files
- Lab objective 21: Generating random large text files
Laboratory objective 1: Working with simple commands
Solution:
- find out what your username is
whoami
alexandru@linux-vje9:~> whoami alexandru
- find out the current date
alexandru@linux-vje9:~> date Thu Aug 31 16:10:52 EEST 2017
- see the calendar
alexandru@linux-vje9:~> cal August 2017 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
- see all the files in your present working directory, sorted based on the time, and show the size of each file too:
ls -lath alexandru@linux-vje9:~> ls -lath total 268K -rw------- 1 alexandru users 17K Aug 31 16:03 .xsession-errors-:0 -rw------- 1 alexandru users 4.4K Aug 31 15:48 .bash_history drwxr-xr-x 31 alexandru users 4.0K Aug 31 15:06 . -rw-r--r-- 1 alexandru users 6.0K Aug 31 15:06 string_grep.txt -rw-r--r-- 1 alexandru users 152K Aug 31 15:06 string.txt drwxr-xr-x 3 alexandru users 4.0K Aug 31 15:00 Downloads drwx------ 6 alexandru users 187 Aug 31 12:23 .dropbox drwxr-xr-x 3 alexandru users 61 Aug 31 12:23 .hplip -rw------- 1 alexandru users 5.6K Aug 31 12:23 .ICEauthority drwxr-xr-x 6 alexandru users 212 Aug 30 17:52 Documents drwxr-xr-x 2 alexandru users 4.0K Aug 30 15:03 Pictures lrwxrwxrwx 1 alexandru users 28 Aug 30 14:27 softlink -> /home/alexandru/TEST/my_file -rw-r--r-- 2 alexandru users 0 Aug 30 14:22 hardlink drwxr-xr-x 2 alexandru users 38 Aug 30 14:22 TEST drwx------ 25 alexandru users 4.0K Aug 30 12:01 .config drwx------ 3 alexandru users 135 Aug 29 22:38 .gnupg drwx------ 19 alexandru users 4.0K Aug 28 19:19 .cache drwx------ 21 alexandru users 4.0K Aug 28 11:16 Dropbox drwxr-xr-x 2 alexandru users 41 Aug 27 15:33 public_html -rw-r--r-- 1 alexandru users 685 Aug 27 12:57 .htaccess drwx------ 3 alexandru users 19 Aug 25 22:16 .pki drwx------ 2 alexandru users 6 Aug 25 17:39 .gnome2 drwx------ 3 alexandru users 26 Aug 25 14:29 .macromedia drwx------ 2 alexandru users 25 Aug 25 11:18 .ssh drwxr-xr-x 5 alexandru users 50 Aug 24 20:41 VirtualBox VMs drwx------ 3 alexandru users 26 Aug 24 17:04 .adobe drwxr-xr-x 2 alexandru users 90 Aug 24 15:53 Videos -rw-r--r-- 1 alexandru users 12K Aug 24 14:44 .y2log drwxr-xr-x 2 alexandru users 20 Aug 24 14:05 .vbox drwx------ 4 alexandru users 39 Aug 24 12:48 .mozilla -rw------- 1 alexandru users 16 Aug 24 12:19 .esd_auth drwx------ 3 alexandru users 19 Aug 24 12:19 .local drwx------ 3 alexandru users 25 Aug 24 12:19 .dbus drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Desktop drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Music drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Public drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Templates drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 .fonts -rw-r--r-- 1 alexandru users 861 Aug 24 12:17 .inputrc -rwxr-xr-x 1 alexandru users 1.1K Aug 24 12:17 .xinitrc.template drwxr-xr-x 3 root root 23 Aug 24 12:17 .. -rw-r--r-- 1 alexandru users 1.2K Aug 24 12:17 .bashrc drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 bin -rw-r--r-- 1 alexandru users 1.6K Aug 24 12:17 .emacs -rw-r--r-- 1 alexandru users 305 Aug 24 12:17 .i18n -rw-r--r-- 1 alexandru users 1.1K Aug 24 12:17 .profile -rw-r--r-- 1 alexandru users 2.0K Aug 24 12:17 .xim.template drwxr-xr-x 3 alexandru users 71 Aug 22 19:34 .dropbox-dist
Laboratory objective 2: Change the character encoding
Solution:
-
open a terminal window
-
to change the character encoding from UTF-8 to POSIX, enter:
LANG=POSIX
- to change back to UTF-8 enter:
LANG=UTF-8
Laboratory objective 3: Use search patterns
Commands used: ls,
Solution:
- list all filenames in the /bin/ directory that start with the character "a" by entering
ls /bin/a
My output:
alexandru@linux-vje9:~> ls /bin/a /bin/arch /bin/awk
- list all file names in the /bin/ directory that consist of four characters by entering:
ls /bin/????
- list all file names in the /bin/ directory that consist of four or more characters by entering:
ls /bin/????*
- list all file names in the /bin/ directory that do not start with one of the characters from a to r by entering:
ls /bin/[!a-r]
My output for exercises 2-4:
alexandru@linux-vje9:~> ls /bin/???? /bin/arch /bin/date /bin/grep /bin/mail /bin/sort /bin/tcsh /bin/bash /bin/echo /bin/gzip /bin/more /bin/stat /bin/true /bin/chvt /bin/find /bin/kill /bin/ping /bin/stty /bin/zcat /bin/cpio /bin/gawk /bin/kmod /bin/sash /bin/sync alexandru@linux-vje9:~> ls /bin/???? /bin/arch /bin/hostname /bin/readlink /bin/basename /bin/initviocons /bin/resizecons /bin/bash /bin/kbdinfo /bin/rmdir /bin/chgrp /bin/kbd_mode /bin/sash /bin/chmod /bin/kbdrate /bin/screendump /bin/chown /bin/keyctl /bin/setfont /bin/chvt /bin/kill /bin/setkeycodes /bin/clrunimap /bin/kmod /bin/setleds /bin/cpio /bin/ksh93 /bin/setlogcons /bin/date /bin/loadkeys /bin/setmetamode /bin/dbus-cleanup-sockets /bin/loadunimap /bin/setpalette /bin/dbus-daemon /bin/logger /bin/setserial /bin/dbus-monitor /bin/login /bin/setvesablank /bin/dbus-send /bin/lsblk /bin/setvtrgb /bin/dbus-uuidgen /bin/lsmod /bin/showconsolefont /bin/deallocvt /bin/mail /bin/showkey /bin/dmesg /bin/mapscrn /bin/sleep /bin/dnsdomainname /bin/md5sum /bin/sort /bin/domainname /bin/mkdir /bin/spawn_console /bin/dumpkeys /bin/mknod /bin/spawn_login /bin/echo /bin/mktemp /bin/stat /bin/egrep /bin/more /bin/stty /bin/false /bin/mount /bin/sync /bin/fgconsole /bin/netstat /bin/systemctl /bin/fgrep /bin/nisdomainname /bin/systemd /bin/fillup /bin/openvt /bin/systemd-ask-password /bin/find /bin/outpsfheader /bin/tcsh /bin/findmnt /bin/pgrep /bin/touch /bin/fsync /bin/pidof /bin/true /bin/fuser /bin/ping /bin/umount /bin/gawk /bin/ping6 /bin/uname /bin/getkeycodes /bin/pkill /bin/unicode_start /bin/getunimap /bin/plymouth /bin/unicode_stop /bin/grep /bin/psfaddtable /bin/usleep /bin/guess_encoding /bin/psfgettable /bin/ypdomainname /bin/gunzip /bin/psfstriptable /bin/zcat /bin/gzip /bin/psfxtable alexandru@linux-vje9:~> ls /bin/[!a-r]* /bin/sash /bin/showkey /bin/touch /bin/screendump /bin/sleep /bin/true /bin/sed /bin/sort /bin/umount /bin/setfont /bin/spawn_console /bin/uname /bin/setkeycodes /bin/spawn_login /bin/unicode_start /bin/setleds /bin/stat /bin/unicode_stop /bin/setlogcons /bin/stty /bin/usleep /bin/setmetamode /bin/su /bin/vi /bin/setpalette /bin/sync /bin/vim /bin/setserial /bin/systemctl /bin/ypdomainname /bin/setvesablank /bin/systemd /bin/zcat /bin/setvtrgb /bin/systemd-ask-password /bin/zsh /bin/sh /bin/tar /bin/showconsolefont /bin/tcsh
Laboratory objective 4: File globbing
Solution:
Using file globbing can help you when working with multiple files and with repetitive tasks. The special characters for file globbing are: * ? ! [ ] -
Globbing characters are NOT the same as regular expressions!!!
- list the files in the current directory using echo and the wildcard character (*) - meaning: show all characters that contain any character, but do not show the ones that start with a dot. Those files can not be handled by the wildcard character.
alexandru@linux-vje9:~> cd /etc alexandru@linux-vje9:/etc> echo *
- list only the files starting with pa and followed by any character
alexandru@linux-vje9:/etc> echo pa* pam.d passwd passwd- passwd.YaST2save
- show all the files that end with a ".d" string and begin with any character
alexandru@linux-vje9:/etc> echo *.d aliases.d apparmor.d auto.master.d bash_completion.d binfmt.d cron.d depmod.d dnsmasq.d dracut.conf.d gdbinit.d grub.d init.d ld.so.conf.d libibverbs.d logrotate.d modprobe.d modules-load.d named.d netconfig.d pam.d permissions.d pptp.d products.d profile.d rc.d request-key.d sane.d sensors.d slp.reg.d smartd_warning.d sudoers.d susehelp.d sysctl.d tmpdirs.d tmpfiles.d xinetd.d zsh_completion.d
- show all the files that start with "li" containing any other character type or number and ending with the .conf string
alexandru@linux-vje9:/etc> echo li*.conf libao.conf libaudit.conf
alexandru@linux-vje9:/etc> ls -l li*.conf -rw-r--r-- 1 root root 21 Aug 24 12:15 libao.conf -rw-r----- 1 root root 191 Jul 7 18:03 libaudit.conf
Laboratory objective 5: Quoting commands
Solution:
- if you have a file that has spaces in its name, here is how you can work with it at the command line:
alexandru@linux-vje9:~> ls -l drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 bin -rw-r--r-- 2 alexandru users 0 Aug 30 14:22 hardlink drwxr-xr-x 2 alexandru users 41 Aug 27 15:33 public_html lrwxrwxrwx 1 alexandru users 28 Aug 30 14:27 softlink -> /home/alexandru/TEST/my_file -rw-r--r-- 1 alexandru users 6141 Aug 31 15:06 string_grep.txt -rw-r--r-- 1 alexandru users 155513 Aug 31 15:06 string.txt -rw-r--r-- 1 alexandru users 0 Aug 31 16:45 Untitled Document 1.txt drwxr-xr-x 2 alexandru users 90 Aug 24 15:53 Videos drwxr-xr-x 5 alexandru users 50 Aug 24 20:41 VirtualBox VMs alexandru@linux-vje9:~> ls Untitled Document 1.txt ls: cannot access 'Untitled': No such file or directory ls: cannot access 'Document': No such file or directory ls: cannot access '1.txt': No such file or directory alexandru@linux-vje9:~> ls Untitled\ Document\ 1.txt Untitled Document 1.txt alexandru@linux-vje9:~>
You will use the "" character to negate the space as a special character and thus tell your Shell that it is part of the file name.
- how to work with a file like: !super!file!.txt? By quoting it:
alexandru@linux-vje9:~> touch !super!file!.txt bash: !super!file!.txt: event not found alexandru@linux-vje9:~> touch '!super!file!.txt' alexandru@linux-vje9:~> ls -l '!super!file!.txt' -rw-r--r-- 1 alexandru users 0 Aug 31 16:50 !super!file!.txt
Attention: $ ' \ ! " those special character do not work inside double quotes!!!
alexandru@linux-vje9:~> touch "!super!file!.txt" bash: !super!file!.txt: event not found alexandru@linux-vje9:~> echo "$SHELL" /bin/bash alexandru@linux-vje9:~> echo '$SHELL' $SHELL
alexandru@linux-vje9:~> echo "This is my: $HOME" This is my: /home/alexandru alexandru@linux-vje9:~> echo 'This is my: $HOME' This is my: $HOME
- working with files tarting with a - (dash)
alexandru@linux-vje9:~> touch '-dashy.txt' touch: invalid date format ‘ashy.txt’ alexandru@linux-vje9:~> touch '\-dashy.txt' alexandru@linux-vje9:~> ls -l '\-dashy.txt' -rw-r--r-- 1 alexandru users 0 Aug 31 17:32 \-dashy.txt alexandru@linux-vje9:~> touch -- -dashy2.txt alexandru@linux-vje9:~> ls -l -- -dashy2.txt -rw-r--r-- 1 alexandru users 0 Aug 31 17:34 -dashy2.txt
Laboratory objective 6: Working with pipes and redirection
Solution:
- pipe the output of the ls command for your home directory to a file
ls ~ > home_dir.txt
- display the content by using
cat home_dir.txt
- append the output of the ls command for the root directory to the home_dir.txt file
ls / >> home_dir.txt
- display the content of the file
cat home_dir.txt
- overwrite the home_dir.txt file with the output of the ls command
ls / > home_dir.txt
- display the content with cat
cat home_dir.txt
- write the output of the ls command on the screen and into the home_dir.txt file using
ls ~ | tee home_dir.txt
- remove the home_dir.txt file
rm home_dir.txt
- verify that the file was removed
ls -l
- create a file with the output of the netstat command, that will show only the opened ports available on your system
netstat -tupan | grep 'LISTEN' > open_ports.txt
My output:
alexandru@linux-vje9:~> netstat -tupan | grep 'LISTEN' (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN - tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:17500 0.0.0.0:* LISTEN 2830/dropbox tcp 0 0 127.0.0.1:17600 0.0.0.0:* LISTEN 2830/dropbox tcp 0 0 127.0.0.1:17603 0.0.0.0:* LISTEN 2830/dropbox tcp 0 0 ::1:631 :::* LISTEN - tcp 0 0 ::1:25 :::* LISTEN - tcp 0 0 :::17500 :::* LISTEN 2830/dropbox alexandru@linux-vje9:~> netstat -tupan | grep 'LISTEN' > open_ports.txt (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) alexandru@linux-vje9:~> cat open_ports.txt tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN - tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:17500 0.0.0.0:* LISTEN 2830/dropbox tcp 0 0 127.0.0.1:17600 0.0.0.0:* LISTEN 2830/dropbox tcp 0 0 127.0.0.1:17603 0.0.0.0:* LISTEN 2830/dropbox tcp 0 0 ::1:631 :::* LISTEN - tcp 0 0 ::1:25 :::* LISTEN - tcp 0 0 :::17500 :::* LISTEN 2830/dropbox
- display only the string that shows the MAC of one of your interfaces, based on the ifconfig output
sudo ifconfig alexandru@linux-vje9:~> sudo ifconfig eth0 | grep 'HW' | cut -d " " -f 11
In some cases, the last string of characters could differ, as it will be based on your ifconfig output. After the first command you should count the number of spaces that are until the MAC address and "cut" them with the cut command. It could be 9, 10 or 11 characters, depending on your system.
You can do the same with your IP address:
alexandru@linux-vje9:~> sudo ifconfig br1 | grep 'inet' inet addr:192.168.0.19 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::f64d:30ff:fe61:4b83/64 Scope:Link
alexandru@linux-vje9:~> sudo ifconfig br1 | grep 'inet' | awk -F'[: ]+' '{ print $4}'
192.168.0.19
fe80
alexandru@linux-vje9:~> ip addr sh | awk '/inet/{print$2}'
127.0.0.1/8
::1/128
fe80::9041:c7ff:feb5:c635/64
192.168.0.19/24
fe80::f64d:30ff:fe61:4b83/64
alexandru@linux-vje9:~> ip a | awk '/inet/{print$2}'
127.0.0.1/8
::1/128
fe80::9041:c7ff:feb5:c635/64
192.168.0.19/24
fe80::f64d:30ff:fe61:4b83/64
Laboratory objective 7: Calculate the overall size of each of your system's top-level directories
Commands used: du
Solution:
In order to calculate the sizes of all your top-level directories, you will use the du command. For this, you should use it in the following state:
sudo du --max-depth=1 -hx /
Options explanations:
--max-depth=1 -> this tells the command to go down one level from / (root) and sum up everything recursively
-h -> give human-readable numbers (KB, MB, GB)
-x -> stay on only one filesystem, do not look on directories that are not on the / (root) partition.
The result on my test system running openSUSE 42.3 is:
alexandru@linux-vje9:~> sudo du --max-depth=1 -hx / [sudo] password for root: 27M /etc 62M /boot 675M /var 4.7G /usr 64K /srv 4.8M /bin 25G /home 634M /lib 18M /lib64 0 /mnt 0 /opt 268K /root 9.6M /sbin 0 /selinux 8.0K /tmp 31G /
This is because I only used one partition for all the filesystems. If I would have used different partitions for / (root) and other, some of the directories listed above would not have appeared, like /bin /sbin /lib or /lib64.
Directories like /dev /proc /run /sys are not listed, because these are pseudo-filesystems which exist in memory only, and are only empty mount points when the system is not running.
Laboratory objective 8: Take a tour of your filesystem and give extra attention to /proc filesystem
Commands used: cd, ls, cat, less
Solution:
As root, place yourself in the /proc directory and do a listing. Do this as root.
sudo su cd /proc ls -F
The result on your terminal will be something similar to the text below. Keep in mind that depending on your distribution and kernel version, your result can be different than mine.
alexandru@linux-vje9:/proc> sudo su linux-vje9:/proc # cd /proc linux-vje9:/proc # ls -F 1/ 17/ 2705/ 2887/ 3047/ 3812/ 5176/ buddyinfo locks 10/ 1788/ 271/ 2892/ 3085/ 3827/ 5177/ bus/ meminfo 1001/ 18/ 2717/ 2897/ 3097/ 3846/ 5178/ cgroups misc 1006/ 1852/ 272/ 29/ 31/ 3853/ 5220/ cmdline modules 1007/ 1913/ 273/ 2902/ 319/ 3898/ 536/ config.gz mounts@ 1013/ 1922/ 274/ 2907/ 32/ 39/ 538/ consoles mtrr 1017/ 2/ 2771/ 2921/ 320/ 3916/ 58/ cpuinfo net@ 1037/ 20/ 2772/ 2926/ 321/ 40/ 59/ crypto pagetypeinfo 1059/ 21/ 2773/ 2929/ 322/ 400/ 61/ devices partitions 1065/ 2158/ 2774/ 2930/ 323/ 42/ 617/ diskstats sched_debug 1087/ 2174/ 2782/ 2935/ 324/ 43/ 62/ dma schedstat 1088/ 2178/ 2787/ 2937/ 3248/ 44/ 7/ driver/ scsi/ 1089/ 22/ 279/ 294/ 325/ 45/ 701/ execdomains self@ 1092/ 2236/ 2796/ 2942/ 326/ 457/ 702/ fb slabinfo 11/ 23/ 28/ 2944/ 327/ 4781/ 712/ filesystems softirqs 1109/ 25/ 280/ 2945/ 328/ 4783/ 717/ fs/ stat 1114/ 2516/ 2801/ 2953/ 33/ 4895/ 72/ interrupts swaps 1145/ 2573/ 2803/ 2954/ 3362/ 4905/ 763/ iomem sys/ 1148/ 2594/ 281/ 2960/ 3411/ 4962/ 769/ ioports sysrq-trigger 1151/ 2611/ 2819/ 298/ 35/ 4968/ 775/ irq/ sysvipc/ 1157/ 2622/ 2827/ 299/ 3546/ 5/ 793/ kallsyms thread-self@ 1160/ 266/ 2831/ 2990/ 36/ 5007/ 8/ kcore timer_list 12/ 267/ 2833/ 2991/ 3623/ 5008/ 823/ key-users timer_stats 13/ 268/ 2838/ 3/ 3632/ 5009/ 9/ keys tty/ 15/ 269/ 2843/ 30/ 3637/ 5011/ 915/ kmsg uptime 16/ 2692/ 2849/ 3024/ 3642/ 5031/ 939/ kpagecgroup version 1636/ 2698/ 286/ 3027/ 37/ 5112/ 965/ kpagecount vmallocinfo 1642/ 27/ 2860/ 3028/ 3731/ 5117/ 967/ kpageflags vmstat 1670/ 270/ 2869/ 3030/ 3743/ 5121/ acpi/ latency_stats zoneinfo 1674/ 2700/ 2882/ 3034/ 38/ 5128/ asound/ loadavg linux-vje9:/proc #
Notice many of the directory names and numbers, as each corresponds to a running process and the name of the directory is the process ID. An important subdirectory is /proc/sys, under which many system parameters can be examined or modified. Take extra care when you explore this one, not to break your system!!!
Explore and view the following files, preferably not being root:
/proc/cpuinfo
/proc/meminfo
/proc/mounts
/proc/swaps
/proc/version
/proc/partitions
/proc/interrupts
In order to view the contents of the files, you will use the cat command, as follows:
cat /proc/version
the result will be:
alexandru@linux-vje9:/proc> cat /proc/version Linux version 4.4.79-19-default (geeko@buildhost) (gcc version 4.8.5 (SUSE Linux) ) #1 SMP Thu Aug 10 20:28:47 UTC 2017 (2dd03e8)
Use the same for every other file. You can also use the command with a pipe and redirect the result to a new command, like less, if the output is to large to view on one screen in your terminal:
cat /proc/cpuinfo | less
This will output the contents of the file one screen at a time, for you to see all the available contents. Press q to exit the less output.
processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 61 model name : Intel(R) Core(TM) i3-5010U CPU @ 2.10GHz stepping : 4 microcode : 0x25 cpu MHz : 1998.988 cache size : 3072 KB physical id : 0 siblings : 4 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 20 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma lines 1-20
Laboratory objective 9: Explore the Linux File system Hierarchy using an installation DVD
Commands used: cd, ls, umount, mount
Solution:
For the purpose of this exercise, we will use an openSUSE installation DVD. You could use any distribution DVD, like CentOS or Debian, for example. If you don't have a cd/dvd drive available, you could do this exercise by using an iso image, by mounting it, but it is more difficult than using a normal DVD.
-
Insert the DVD into your drive
-
see the contents of the DVD by accessing run/media/user/mountpoint directory, where mountpoint is the mount point of the DVD, which you will have to discover first, and user is your user name, which in my case is "alexandru". In order to do this, run the command:
cd /run/media/alexandru/ ls
Now, for each distribution, here is the output.
CentOS 7
alexandru@linux-vje9:~> cd /run/media/alexandru/ alexandru@linux-vje9:/run/media/alexandru> ls CentOS 7 x86_64 FE46CB7446CB2BE9 alexandru@linux-vje9:/run/media/alexandru> cd CentOS\ 7\ x86_64/ alexandru@linux-vje9:/run/media/alexandru/CentOS 7 x86_64> ls CentOS_BuildTag GPL LiveOS RPM-GPG-KEY-CentOS-7 EFI images Packages RPM-GPG-KEY-CentOS-Testing-7 EULA isolinux repodata TRANS.TBL
openSUSE 42.3
alexandru@linux-vje9:/run/media/alexandru> cd openSUSE-Leap-42.3-DVD-x86_64033/ alexandru@linux-vje9:/run/media/alexandru/openSUSE-Leap-42.3-DVD-x86_64033> ls ARCHIVES.gz docu media.1 autorun.inf EFI openSUSE_installer.exe boot gpg-pubkey-307e3d54-4be01a65.asc README ChangeLog gpg-pubkey-3dbdc284-53674dd4.asc suse content GPLv2.txt SuSEgo.ico content.asc GPLv3.txt SuSEgo.png content.key INDEX.gz SuSEgo.svg control.xml license.tar.gz directory.yast ls-lR.gz
Debian 9.1
alexandru@linux-vje9:/run/media/alexandru> cd Debian\ 9.1.0\ amd64\ 1/ alexandru@linux-vje9:/run/media/alexandru/Debian 9.1.0 amd64 1> ls autorun.inf doc install pool README.txt boot efi install.amd README.html setup.exe css firmware isolinux README.mirrors.html tools debian g2ldr md5sum.txt README.mirrors.txt win32-loader.ini dists g2ldr.mbr pics README.source
Explor each of the distributions and see what the DVD contain.
- unmount each dvd with the commands:
sudo umount /run/media/alexandru/Debian\ 9.1.0\ amd64\ 1 sudo umount /run/media/alexandru/openSUSE-Leap-42.3-DVD-x86_64033 sudo umount /run/media/alexandru/CentOS\ 7\ x86_64/
- now mount one of the dvd's at the /mnt point, manually with the command:
sudo mount /dev/sr0 /mnt
In openSUSE, the unmounted dvd can still be accessible from /dev/sr0.
- now explore the new mount point with the command:
ls /mnt
- now unmount using the command:
sudo umount /mnt
Laboratory objective 10: Change directories and list directory contents
Commands used: pwd, cd, ls
Solution:
During this exercise, you will work with pwd, cd and ls commands. For this, do the following.
- in your terminal, type
pwd
this will output your present working directory, which should be /home/user, where "user" is your username, like in my case "alexandru"
- now type the command
cd /tmp
this will change the working directory to /tmp. You can check using the command
pwd
- now use the command
cd /usr/share/doc
this will change the directory inside /usr/share/doc
- now type
cd
this will move you to the /home/user directory, which is your home directory in Linux
- now play with the ls command like this:
ls ls -a ls -la ls -alh
compare the outputs of the ls command.
You can do this exercise with any directory you like, not just with the ones described here.
Laboratory objective 11: Create and view files
Commands used: touch, cat, less, head, tail
Solution:
You will create and view files.
- open your terminal and create a file using the touch command:
touch new_file
this file will be create in your present working directory, which is /home/username.
- open a new terminal and log in as root with the command:
su -
enter your password
- display the content of a log file
Now, if you run openSUSE, you will notice that there is no more /var/log/messages, as logging is now the job of systemd, through journalctl. But if you still want to see you log file in the old fashion way, that is from /var/log/messages, you need to install rsyslog. Do this with the command sudo zypper in rsyslog, but it will ask you to uninstall systemd-logger. It is your choice.
Now, on CentOS, you can still use /var/log/messages, thus we will use it for this example. To view the /var/log/messages, do the command:
sudo cat /var/log/messages | less
- you can search for text in the file
while running the less command, enter
/root
to search for the firs instance of the word root in the file
-
to find the next occurrence of the word root, hit n. Exit from less hitting q.
-
to view a continuous updated display of the last line of the file /var/log/messages enter:
sudo tail -f /var/log/messages
-
from a new terminal window, login as root and try to enter a wrong password
-
while doing this, notice how is this shown on the first terminal window, where you are looking at the last lines of /var/log/messages in real time.
-
stop the tail command using ctrl+c and exit to exit the terminal
Laboratory objective 12: Perform multiple file operations
Commands used: ls, mkdir, cp, mv, rm, rmdir, ln
Solution:
Part 1. Copy and move files and directories
- open a terminal and rename the file you created in your previous exercise:
mv new_file my_file
- verify if the file was renamed using the ls command:
ls -l
- make a copy of the renamed file, called my_file_1
cp my_file my_file_1
- you can verify if my_file_1 was created with the ls command
ls -l my*
- copy the /usr/bin/rename and /usr/bin/tac files to the /tmp directory using the commands:
cp /usr/bin/rename /usr/bin/tac /tmp
- verify if the files have been copied using the command:
ls -l /tmp
- move the /tmp/tac file to your home directory
mv /tmp/tac ~
-
verify by using the ls -l command
-
move and rename the /tmp/rename file to ~/my_file2 by using the command:
mv /tmp/rename ~/my_file2
-
verify with the ls -l command
-
copy the complete /bin/ directory to the home directory with the new directory called my_dir using the command
cp -r /bin ~/my_dir
-
verify the process with ls -l command
-
now, remove all the files and directories that you created
Here is an output of all the commands, on my openSUSE system:
alexandru@linux-vje9:~> touch new_file alexandru@linux-vje9:~> ls bin Documents Dropbox new_file Public Templates VirtualBox VMs Desktop Downloads Music Pictures public_html Videos alexandru@linux-vje9:~> mv new_file my_file alexandru@linux-vje9:~> ls bin Documents Dropbox my_file Public Templates VirtualBox VMs Desktop Downloads Music Pictures public_html Videos alexandru@linux-vje9:~> cp my_file my_file_1 alexandru@linux-vje9:~> ls bin Documents Dropbox my_file Pictures public_html Videos Desktop Downloads Music my_file_1 Public Templates VirtualBox VMs alexandru@linux-vje9:~> ls -l my* -rw-r--r-- 1 alexandru users 0 Aug 29 23:05 my_file -rw-r--r-- 1 alexandru users 0 Aug 29 23:06 my_file_1 alexandru@linux-vje9:~> cp /usr/bin/rename /usr/bin/tac /tmp alexandru@linux-vje9:~> ls -l /tmp total 52 drwx------ 2 alexandru users 6 Aug 29 22:52 firefox_alexandru -rwxr-xr-x 1 alexandru users 14360 Aug 29 23:06 rename drwx------ 3 root root 17 Aug 29 10:51 systemd-private-38cece39fba54ce3b6137d566ab6fb8d-colord.service-Z2CUH1 drwx------ 3 root root 17 Aug 29 10:51 systemd-private-38cece39fba54ce3b6137d566ab6fb8d-ntpd.service-yrTjIy drwx------ 3 root root 17 Aug 29 10:51 systemd-private-38cece39fba54ce3b6137d566ab6fb8d-rtkit-daemon.service-X9B2XP -rwxr-xr-x 1 alexandru users 35680 Aug 29 23:06 tac alexandru@linux-vje9:~> mv /tmp/tac ~ alexandru@linux-vje9:~> ls bin Downloads my_file Public Templates Desktop Dropbox my_file_1 public_html Videos Documents Music Pictures tac VirtualBox VMs alexandru@linux-vje9:~> mv /tmp/rename ~/my_file2 alexandru@linux-vje9:~> ls -l total 68 drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 bin drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Desktop drwxr-xr-x 6 alexandru users 4096 Aug 29 23:05 Documents drwxr-xr-x 16 alexandru users 4096 Aug 28 22:42 Downloads drwx------ 21 alexandru users 4096 Aug 28 11:16 Dropbox drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Music -rw-r--r-- 1 alexandru users 0 Aug 29 23:05 my_file -rw-r--r-- 1 alexandru users 0 Aug 29 23:06 my_file_1 -rwxr-xr-x 1 alexandru users 14360 Aug 29 23:06 my_file2 drwxr-xr-x 2 alexandru users 4096 Aug 27 12:24 Pictures drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Public drwxr-xr-x 2 alexandru users 41 Aug 27 15:33 public_html -rwxr-xr-x 1 alexandru users 35680 Aug 29 23:06 tac drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Templates drwxr-xr-x 2 alexandru users 90 Aug 24 15:53 Videos drwxr-xr-x 5 alexandru users 50 Aug 24 20:41 VirtualBox VMs alexandru@linux-vje9:~> cp -r /bin ~/my_dir alexandru@linux-vje9:~> ls bin Downloads my_dir my_file2 public_html Videos Desktop Dropbox my_file Pictures tac VirtualBox VMs Documents Music my_file_1 Public Templates alexandru@linux-vje9:~> ls -l total 76 drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 bin drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Desktop drwxr-xr-x 6 alexandru users 4096 Aug 29 23:05 Documents drwxr-xr-x 16 alexandru users 4096 Aug 28 22:42 Downloads drwx------ 21 alexandru users 4096 Aug 28 11:16 Dropbox drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Music drwxr-xr-x 2 alexandru users 4096 Aug 29 23:07 my_dir -rw-r--r-- 1 alexandru users 0 Aug 29 23:05 my_file -rw-r--r-- 1 alexandru users 0 Aug 29 23:06 my_file_1 -rwxr-xr-x 1 alexandru users 14360 Aug 29 23:06 my_file2 drwxr-xr-x 2 alexandru users 4096 Aug 27 12:24 Pictures drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Public drwxr-xr-x 2 alexandru users 41 Aug 27 15:33 public_html -rwxr-xr-x 1 alexandru users 35680 Aug 29 23:06 tac drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Templates drwxr-xr-x 2 alexandru users 90 Aug 24 15:53 Videos drwxr-xr-x 5 alexandru users 50 Aug 24 20:41 VirtualBox VMs alexandru@linux-vje9:~> rm -r my_dir/ alexandru@linux-vje9:~> ls bin Downloads my_file Pictures tac VirtualBox VMs Desktop Dropbox my_file_1 Public Templates Documents Music my_file2 public_html Videos alexandru@linux-vje9:~> ls -l total 68 drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 bin drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Desktop drwxr-xr-x 6 alexandru users 4096 Aug 29 23:05 Documents drwxr-xr-x 16 alexandru users 4096 Aug 28 22:42 Downloads drwx------ 21 alexandru users 4096 Aug 28 11:16 Dropbox drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Music -rw-r--r-- 1 alexandru users 0 Aug 29 23:05 my_file -rw-r--r-- 1 alexandru users 0 Aug 29 23:06 my_file_1 -rwxr-xr-x 1 alexandru users 14360 Aug 29 23:06 my_file2 drwxr-xr-x 2 alexandru users 4096 Aug 27 12:24 Pictures drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Public drwxr-xr-x 2 alexandru users 41 Aug 27 15:33 public_html -rwxr-xr-x 1 alexandru users 35680 Aug 29 23:06 tac drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Templates drwxr-xr-x 2 alexandru users 90 Aug 24 15:53 Videos drwxr-xr-x 5 alexandru users 50 Aug 24 20:41 VirtualBox VMs alexandru@linux-vje9:~> rm my_file my_file_1 my_file2 tac alexandru@linux-vje9:~> ls -l total 16 drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 bin drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Desktop drwxr-xr-x 6 alexandru users 4096 Aug 29 23:05 Documents drwxr-xr-x 16 alexandru users 4096 Aug 28 22:42 Downloads drwx------ 21 alexandru users 4096 Aug 28 11:16 Dropbox drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Music drwxr-xr-x 2 alexandru users 4096 Aug 27 12:24 Pictures drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Public drwxr-xr-x 2 alexandru users 41 Aug 27 15:33 public_html drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Templates drwxr-xr-x 2 alexandru users 90 Aug 24 15:53 Videos drwxr-xr-x 5 alexandru users 50 Aug 24 20:41 VirtualBox VMs
Part 2: Create directories
- create an empty directory that is inside a new directory with the command
mkdir ~/my_dir/new_dir
-
verify that the directory has been created with the ls -l command
-
create a new directory and subdirectory with the command
mkdir -p ~/geeko_dir/empty_dir
- verify using the ls -l command
Investigate the errors that you encounter. You will see that you can not create a directory inside another directory without the option -p. This option specifies the command that the newly created directory is inside a parent directory.
Here is the output from my system:
alexandru@linux-vje9:~> mkdir ~/my_dir/new_dir mkdir: cannot create directory ‘/home/alexandru/my_dir/new_dir’: No such file or directory alexandru@linux-vje9:~> mkdir -p ~/geeko_dir/empty_dir alexandru@linux-vje9:~> ls -l total 16 drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 bin drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Desktop drwxr-xr-x 6 alexandru users 4096 Aug 29 23:05 Documents drwxr-xr-x 16 alexandru users 4096 Aug 28 22:42 Downloads drwx------ 21 alexandru users 4096 Aug 28 11:16 Dropbox drwxr-xr-x 3 alexandru users 23 Aug 30 11:26 geeko_dir drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Music drwxr-xr-x 2 alexandru users 4096 Aug 27 12:24 Pictures drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Public drwxr-xr-x 2 alexandru users 41 Aug 27 15:33 public_html drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Templates drwxr-xr-x 2 alexandru users 90 Aug 24 15:53 Videos drwxr-xr-x 5 alexandru users 50 Aug 24 20:41 VirtualBox VMs alexandru@linux-vje9:~> cd geeko_dir/ alexandru@linux-vje9:~/geeko_dir> ls -l total 0 drwxr-xr-x 2 alexandru users 6 Aug 30 11:26 empty_dir
Another solution is to use mkdir, cd and ls commands and create the directories one at a time. This is how:
alexandru@linux-vje9:~> mkdir ~/my_dir alexandru@linux-vje9:~> cd my_dir/ alexandru@linux-vje9:~/my_dir> ls alexandru@linux-vje9:~/my_dir> mkdir new_dir alexandru@linux-vje9:~/my_dir> ls new_dir alexandru@linux-vje9:~/my_dir> cd.. alexandru@linux-vje9:~> ls -l total 16 drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 bin drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Desktop drwxr-xr-x 6 alexandru users 4096 Aug 29 23:05 Documents drwxr-xr-x 16 alexandru users 4096 Aug 28 22:42 Downloads drwx------ 21 alexandru users 4096 Aug 28 11:16 Dropbox drwxr-xr-x 3 alexandru users 23 Aug 30 11:26 geeko_dir drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Music drwxr-xr-x 3 alexandru users 21 Aug 30 11:33 my_dir drwxr-xr-x 2 alexandru users 4096 Aug 27 12:24 Pictures drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Public drwxr-xr-x 2 alexandru users 41 Aug 27 15:33 public_html drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Templates drwxr-xr-x 2 alexandru users 90 Aug 24 15:53 Videos drwxr-xr-x 5 alexandru users 50 Aug 24 20:41 VirtualBox VMs alexandru@linux-vje9:~>
Part 3. Delete files and directories
- try to remove the directory geeko_dir with the rmdir command, assuming that you are already in your home directory
rmdir geeko_dir
you will receive an error that says the directory can not be removed, because it is not empty
- use the rmdir command in this way
rmdir geeko_dir/empty_dir
this will remove the subdirectory called empty_dir, which is inside the geeko_dir directory. Check using the ls command:
alexandru@linux-vje9:~> rmdir geeko_dir rmdir: failed to remove 'geeko_dir': Directory not empty alexandru@linux-vje9:~> rmdir geeko_dir/empty_dir alexandru@linux-vje9:~> ls -l geeko_dir
- remove the geeko_dir with the rmdir command
rmdir geeko_dir
-
verify with the ls command
-
remove the my_dir/new_dir directory with the rmdir command using the -r option
rm -r ~/my_dir/new_dir
The output on my system is
alexandru@linux-vje9:~> ls bin Documents Dropbox Music Pictures public_html Videos Desktop Downloads geeko_dir my_dir Public Templates VirtualBox VMs alexandru@linux-vje9:~> rm -r my_dir alexandru@linux-vje9:~> ls bin Documents Dropbox Music Public Templates VirtualBox VMs Desktop Downloads geeko_dir Pictures public_html Videos alexandru@linux-vje9:~> rmdir geeko_dir/ alexandru@linux-vje9:~> ls bin Documents Dropbox Pictures public_html Videos Desktop Downloads Music Public Templates VirtualBox VMs alexandru@linux-vje9:~>
Part 4. Link files
In this exercise you will create hard and symbolic links to some files.
For the purpose of this exercise we will create some new directories and files. I will create a directory called TEST inside my home directory, and in that directory I will create two files, called my_file and my_file_1. This is the output:
alexandru@linux-vje9:~> ls bin Documents Dropbox Pictures public_html Videos Desktop Downloads Music Public Templates VirtualBox VMs alexandru@linux-vje9:~> mkdir TEST alexandru@linux-vje9:~> ls bin Documents Dropbox Pictures public_html TEST VirtualBox VMs Desktop Downloads Music Public Templates Videos alexandru@linux-vje9:~> cd TEST/ alexandru@linux-vje9:~/TEST> ls alexandru@linux-vje9:~/TEST> touch my_file alexandru@linux-vje9:~/TEST> touch my_file_1 alexandru@linux-vje9:~/TEST> ls my_file my_file_1 alexandru@linux-vje9:~/TEST>
- create a symbolic link of the file my_file from the TEST directory, inside your home directory
ln -s ~/TEST/my_file softlink
- create a hard link of the file my_file_1 from the TEST directory, inside your home directory
ln ~/TEST/my_file_1 hardlink
- display the links by entering ls -l command in your home directory
Notice that the symbolic link shows the file that it links to.
Output:
alexandru@linux-vje9:~/TEST> cd.. alexandru@linux-vje9:~> pwd /home/alexandru alexandru@linux-vje9:~> ln -s ~/TEST/my_file softlink alexandru@linux-vje9:~> ln ~/TEST/my_file_1 hardlink alexandru@linux-vje9:~> ls -l total 16 drwxr-xr-x 2 alexandru users 6 Aug 24 12:17 bin drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Desktop drwxr-xr-x 6 alexandru users 4096 Aug 29 23:05 Documents drwxr-xr-x 16 alexandru users 4096 Aug 28 22:42 Downloads drwx------ 21 alexandru users 4096 Aug 28 11:16 Dropbox -rw-r--r-- 2 alexandru users 0 Aug 30 14:22 hardlink drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Music drwxr-xr-x 2 alexandru users 4096 Aug 27 12:24 Pictures drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Public drwxr-xr-x 2 alexandru users 41 Aug 27 15:33 public_html lrwxrwxrwx 1 alexandru users 28 Aug 30 14:27 softlink -> /home/alexandru/TEST/my_file drwxr-xr-x 2 alexandru users 6 Aug 24 12:19 Templates drwxr-xr-x 2 alexandru users 38 Aug 30 14:22 TEST drwxr-xr-x 2 alexandru users 90 Aug 24 15:53 Videos drwxr-xr-x 5 alexandru users 50 Aug 24 20:41 VirtualBox VMs alexandru@linux-vje9:~>
Laboratory objective 13: Find files in Linux
Commands used: type, whereis, which, find
Solution:
In this exercise you will search for files using the command line.
- find the type of the ll command by entering
type ll
- find the manual pages of the find command by entering
whereis -m find
- find the path of the program Firefox by entering
which firefox
The output is:
alexandru@linux-vje9:~> type ll ll is aliased to `ls -l' alexandru@linux-vje9:~> whereis -m find find: /usr/share/man/man1/find.1.gz /usr/share/man/man1p/find.1p.gz /usr/share/man/mann/find.n.gz /usr/share/info/find.info-1.gz /usr/share/info/find.info-2.gz /usr/share/info/find.info.gz alexandru@linux-vje9:~> which firefox /usr/bin/firefox
- find all files in the home directory whose names start with "my" by entering the following
find ~ -name "my*"
- find all files in the /tmp/ directory that were changed or created in the last 24 hours by entering the following
find /tmp -ctime -1
The output is:
alexandru@linux-vje9:~> find ~ -name "my*" /home/alexandru/.config/libreoffice/4-suse/user/autotext/mytexts.bau /home/alexandru/Documents/20170828_openlark_backup/public_html/wp-content/plugins/wordfence/images/flags/my.png /home/alexandru/Documents/20170828_openlark_backup/public_html/drupal/core/lib/Drupal/Core/Database/Driver/mysql /home/alexandru/Documents/20170828_openlark_backup/public_html/wp-admin/my-sites.php /home/alexandru/TEST/my_file /home/alexandru/TEST/my_file_1 alexandru@linux-vje9:~> find /tmp -ctime -1 /tmp /tmp/.X11-unix /tmp/.X11-unix/X0 /tmp/.ICE-unix /tmp/.ICE-unix/2342 /tmp/.ICE-unix/2466 /tmp/.ICE-unix/2703 /tmp/.esd-478 find: ‘/tmp/.esd-478’: Permission denied /tmp/firefox_alexandru /tmp/firefox_alexandru/.parentlock /tmp/.esd-1000 /tmp/.esd-1000/socket /tmp/hsperfdata_alexandru /tmp/mozilla_alexandru0 /tmp/systemd-private-4a9a7825abfd4ca6bd9c465bfbc66b9f-ntpd.service-c7XwWy find: ‘/tmp/systemd-private-4a9a7825abfd4ca6bd9c465bfbc66b9f-ntpd.service-c7XwWy’: Permission denied /tmp/systemd-private-4a9a7825abfd4ca6bd9c465bfbc66b9f-colord.service-DxMlbD find: ‘/tmp/systemd-private-4a9a7825abfd4ca6bd9c465bfbc66b9f-colord.service-DxMlbD’: Permission denied /tmp/.X0-lock /tmp/systemd-private-4a9a7825abfd4ca6bd9c465bfbc66b9f-rtkit-daemon.service-zXfS3l find: ‘/tmp/systemd-private-4a9a7825abfd4ca6bd9c465bfbc66b9f-rtkit-daemon.service-zXfS3l’: Permission denied /tmp/ssh-5E63WzjzlUY6 /tmp/ssh-5E63WzjzlUY6/agent.2703 /tmp/gpg-QhF4IU /tmp/gpg-QhF4IU/S.gpg-agent /tmp/tracker-extract-files.1000
Laboratory objective 14: Search file content
Commands used: grep, egrep, ls
Solution:
For the purpose of this exercise, I use openSUSE 42.3, thus there will be specific files found only on SUSE systems.
- create a file called "string.txt" with the output of the ls -l command on /usr/bin directory. In the resulted file, search for all the lines containing the "pam" string, and then create a new file with the output of grep.
ls -l /usr/bin > string.txt grep "pam" string.txt grep "pam" string.txt > string_grep.txt
The output in my case is:
alexandru@linux-vje9:~> ls -l /usr/bin > string.txt alexandru@linux-vje9:~> grep "pam" string.txt -rwxr-xr-x 1 root root 10520 Jun 26 07:40 avstopam -rwxr-xr-x 1 root root 96936 Jun 26 07:40 cameratopam -rwxr-xr-x 1 root root 10568 Jun 26 07:40 hdifftopam -rwxr-xr-x 1 root root 14712 Jun 26 07:40 infotopam -rwxr-xr-x 1 root root 14776 Jun 26 07:40 jpeg2ktopam -rwxr-xr-x 1 root root 14720 Jun 26 07:40 pamaddnoise -rwxr-xr-x 1 root root 14704 Jun 26 07:40 pamarith -rwxr-xr-x 1 root root 10648 Jun 26 07:40 pambackground -rwxr-xr-x 1 root root 10536 Jun 26 07:40 pambayer -rwxr-xr-x 1 root root 10592 Jun 26 07:40 pamchannel -rwxr-xr-x 1 root root 18864 Jun 26 07:40 pamcomp -rwxr-xr-x 1 root root 14744 Jun 26 07:40 pamcut -rwxr-xr-x 1 root root 10568 Jun 26 07:40 pamdeinterlace -rwxr-xr-x 1 root root 10584 Jun 26 07:40 pamdepth -rwxr-xr-x 1 root root 10592 Jun 26 07:40 pamdice -rwxr-xr-x 1 root root 18832 Jun 26 07:40 pamditherbw -rwxr-xr-x 1 root root 10552 Jun 26 07:40 pamedge -rwxr-xr-x 1 root root 6408 Jun 26 07:40 pamendian -rwxr-xr-x 1 root root 10600 Jun 26 07:40 pamenlarge -rwxr-xr-x 1 root root 10616 Jun 26 07:40 pamexec -rwxr-xr-x 1 root root 10576 Jun 26 07:40 pamfile -rwxr-xr-x 1 root root 10608 Jun 26 07:40 pamfix -rwxr-xr-x 1 root root 2062 Jun 26 07:40 pamfixtrunc -rwxr-xr-x 1 root root 18864 Jun 26 07:40 pamflip -rwxr-xr-x 1 root root 14680 Jun 26 07:40 pamfunc -rwxr-xr-x 1 root root 10560 Jun 26 07:40 pamgauss -rwxr-xr-x 1 root root 10560 Jun 26 07:40 pamgradient -rwxr-xr-x 1 root root 10632 Jun 26 07:40 pamlookup -rwxr-xr-x 1 root root 10560 Jun 26 07:40 pammasksharpen -rwxr-xr-x 1 root root 10664 Jun 26 07:40 pammixinterlace -rwxr-xr-x 1 root root 10576 Jun 26 07:40 pammosaicknit -rwxr-xr-x 1 root root 10568 Jun 26 07:40 pamoil lrwxrwxrwx 1 root root 5 Aug 24 12:23 pamon -> pacat -rwxr-xr-x 1 root root 10648 Jun 26 07:40 pampaintspill -rwxr-xr-x 1 root root 22912 Jun 26 07:40 pamperspective -rwxr-xr-x 1 root root 10584 Jun 26 07:40 pampick -rwxr-xr-x 1 root root 10536 Jun 26 07:40 pampop9 -rwxr-xr-x 1 root root 14712 Jun 26 07:40 pamrecolor -rwxr-xr-x 1 root root 19152 Jun 26 07:40 pamrgbatopng -rwxr-xr-x 1 root root 35200 Jun 26 07:40 pamrubber -rwxr-xr-x 1 root root 27128 Jun 26 07:40 pamscale -rwxr-xr-x 1 root root 10544 Jun 26 07:40 pamseq -rwxr-xr-x 1 root root 10552 Jun 26 07:40 pamsharpmap -rwxr-xr-x 1 root root 10544 Jun 26 07:40 pamsharpness -rwxr-xr-x 1 root root 10608 Jun 26 07:40 pamsistoaglyph -rwxr-xr-x 1 root root 10528 Jun 26 07:40 pamslice -rwxr-xr-x 1 root root 10624 Jun 26 07:40 pamsplit -rwxr-xr-x 1 root root 10568 Jun 26 07:40 pamstack -rwxr-xr-x 1 root root 23000 Jun 26 07:40 pamstereogram -rwxr-xr-x 1 root root 10576 Jun 26 07:40 pamstretch -rwxr-xr-x 1 root root 2184 Jun 26 07:40 pamstretch-gen -rwxr-xr-x 1 root root 10536 Jun 26 07:40 pamsumm -rwxr-xr-x 1 root root 10568 Jun 26 07:40 pamsummcol -rwxr-xr-x 1 root root 14760 Jun 26 07:40 pamthreshold -rwxr-xr-x 1 root root 14680 Jun 26 07:40 pamtilt -rwxr-xr-x 1 root root 10528 Jun 26 07:40 pamtoavs -rwxr-xr-x 1 root root 10616 Jun 26 07:40 pamtodjvurle -rwxr-xr-x 1 root root 10560 Jun 26 07:40 pamtofits -rwxr-xr-x 1 root root 18952 Jun 26 07:40 pamtogif -rwxr-xr-x 1 root root 10552 Jun 26 07:40 pamtohdiff -rwxr-xr-x 1 root root 10600 Jun 26 07:40 pamtohtmltbl -rwxr-xr-x 1 root root 18936 Jun 26 07:40 pamtojpeg2k -rwxr-xr-x 1 root root 10576 Jun 26 07:40 pamtompfont -rwxr-xr-x 1 root root 10624 Jun 26 07:40 pamtooctaveimg -rwxr-xr-x 1 root root 10520 Jun 26 07:40 pamtopam -rwxr-xr-x 1 root root 18840 Jun 26 07:40 pamtopdbimg -rwxr-xr-x 1 root root 10544 Jun 26 07:40 pamtopfm -rwxr-xr-x 1 root root 10552 Jun 26 07:40 pamtopnm -rwxr-xr-x 1 root root 18856 Jun 26 07:40 pamtosrf -rwxr-xr-x 1 root root 68040 Jun 26 07:40 pamtosvg -rwxr-xr-x 1 root root 14752 Jun 26 07:40 pamtotga -rwxr-xr-x 1 root root 31248 Jun 26 07:40 pamtotiff -rwxr-xr-x 1 root root 10616 Jun 26 07:40 pamtouil -rwxr-xr-x 1 root root 18944 Jun 26 07:40 pamtowinicon -rwxr-xr-x 1 root root 10600 Jun 26 07:40 pamtoxvmini -rwxr-xr-x 1 root root 14720 Jun 26 07:40 pamundice -rwxr-xr-x 1 root root 10624 Jun 26 07:40 pamvalidate -rwxr-xr-x 1 root root 10592 Jun 26 07:40 pamwipeout -rwxr-xr-x 1 root root 35616 Jun 26 07:40 pamx -rwxr-xr-x 1 root root 18896 Jun 26 07:40 pdbimgtopam -rwxr-xr-x 1 root root 10576 Jun 26 07:40 pfmtopam lrwxrwxrwx 1 root root 7 Aug 24 12:10 pgmedge -> pamedge lrwxrwxrwx 1 root root 6 Aug 24 12:10 pgmoil -> pamoil lrwxrwxrwx 1 root root 8 Aug 24 12:10 pgmslice -> pamslice -rwxr-xr-x 1 root root 31536 Jun 26 07:40 pngtopam lrwxrwxrwx 1 root root 8 Aug 24 12:10 pngtopnm -> pngtopam lrwxrwxrwx 1 root root 8 Aug 24 12:10 pnmarith -> pamarith lrwxrwxrwx 1 root root 6 Aug 24 12:10 pnmcut -> pamcut lrwxrwxrwx 1 root root 8 Aug 24 12:10 pnmdepth -> pamdepth lrwxrwxrwx 1 root root 10 Aug 24 12:10 pnmenlarge -> pamenlarge lrwxrwxrwx 1 root root 7 Aug 24 12:10 pnmfile -> pamfile lrwxrwxrwx 1 root root 10 Aug 24 12:10 pnminterp -> pamstretch lrwxrwxrwx 1 root root 8 Aug 24 12:10 pnmscale -> pamscale lrwxrwxrwx 1 root root 8 Aug 24 12:10 pnmsplit -> pamsplit lrwxrwxrwx 1 root root 9 Aug 24 12:10 pnmtofits -> pamtofits lrwxrwxrwx 1 root root 8 Aug 24 12:10 pnmtopnm -> pamtopnm lrwxrwxrwx 1 root root 9 Aug 24 12:10 pnmtotiff -> pamtotiff lrwxrwxrwx 1 root root 8 Aug 24 12:10 ppmtouil -> pamtouil -rwxr-xr-x 1 root root 10584 Jun 26 07:40 rlatopam -rwxr-xr-x 1 root root 14744 Jun 26 07:40 srftopam -rwxr-xr-x 1 root root 18976 Jun 26 07:40 svgtopam -rwxr-xr-x 1 root root 18824 Jun 26 07:40 winicontopam alexandru@linux-vje9:~> grep "pam" string.txt > string_grep.txt
- Find all HTML headings of hierarchy 2 in the /usr/share/doc/packages/yast2-users/users.html file.
alexandru@linux-vje9:~> cd /usr/share/doc/packages/yast2-users/ alexandru@linux-vje9:/usr/share/doc/packages/yast2-users> ls COPYING users.html alexandru@linux-vje9:/usr/share/doc/packages/yast2-users> grep "<h2>" users.html <h2>Features (SL 9.3)</h2> <h2>Implementation</h2> <h2>The files</h2> alexandru@linux-vje9:/usr/share/doc/packages/yast2-users>
- Find all locations in the HTML files of the same directory that include the word "configuration" by entering
grep configuration /usr/share/doc/packages/yast2-users/*.html alexandru@linux-vje9:/usr/share/doc/packages/yast2-users> grep configuration *.html <h1>YaST2: Users configuration module</h1>
- Find all locations in the HTML files of all "yast2" directories /usr/share/doc/packages/yast2-*/ that include lines beginning with a number or by letter "m" by entering
alexandru@linux-vje9:/usr/share/doc/packages/yast2-users> egrep "^[0-9]" /usr/share/doc/packages/yast2-*/*.html alexandru@linux-vje9:/usr/share/doc/packages/yast2-users> egrep "^[m]" /usr/share/doc/packages/yast2-*/*.html /usr/share/doc/packages/yast2-hardware-detection/hwprobe.html:matching a specific feature. All these entries have a common layout alexandru@linux-vje9:/usr/share/doc/packages/yast2-users>
Laboratory objective 15: Using the tar utility
Commands used: tar, ls, mkdir, gzip, bzip2, xz
Solution:
For the purpose of this exercise we will get you through a series of sub-exercises on how to work with archives, specially using the tar command and the compression utilities for gzip, bzip2 and xz.
- group all the files in the current working directory and compress the resulting bundle with gzip, bzip2 and xz.
for gzip:
tar cvzf my1.tar.gz file[0-9]
for bzip2:
tar cvjf my2.tar.bz2 file[0-9]
for xz:
tar cvJf my3.tar.xz file[0-9]
In order to do this, we will create some files in the /tar directory inside home directory. There will be 9 files with different content and sizes. We will need this in order to compare the rezulted compressed file sizes.
the output in my case:
alexandru@linux-ampu:~> ls bin Documents Dropbox Pictures public_html Videos Desktop Downloads Music Public Templates VirtualBox VMs alexandru@linux-ampu:~> mkdir tar alexandru@linux-ampu:~> ls bin Documents Dropbox Pictures public_html Templates VirtualBox VMs Desktop Downloads Music Public tar Videos alexandru@linux-ampu:~> cd tar/ alexandru@linux-ampu:~/tar> ls alexandru@linux-ampu:~/tar> touch file0 alexandru@linux-ampu:~/tar> touch file1 file2 file3 file4 file5 alexandru@linux-ampu:~/tar> ls file0 file1 file2 file3 file4 file5 alexandru@linux-ampu:~/tar> touch file6 file7 file8 file9 alexandru@linux-ampu:~/tar> ls file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 alexandru@linux-ampu:~/tar> ls -l /usr > file0 alexandru@linux-ampu:~/tar> cat file0 total 4 drwxr-xr-x 1 root root 38456 Sep 2 18:09 bin drwxr-xr-x 1 root root 0 May 10 02:26 games drwxr-xr-x 1 root root 2556 Sep 2 18:08 include drwxr-xr-x 1 root root 6504 Sep 2 18:09 lib drwxr-xr-x 1 root root 81898 Sep 2 18:47 lib64 drwxr-xr-x 1 root root 76 Sep 2 17:20 local drwxr-xr-x 1 root root 11574 Sep 2 18:09 sbin drwxr-xr-x 1 root root 3678 Sep 2 18:09 share drwxr-xr-x 1 root root 140 Sep 2 18:09 src lrwxrwxrwx 1 root root 10 Sep 2 17:20 tmp -> ../var/tmp drwxr-xr-x 1 root root 10 Sep 2 17:20 X11R6 drwxr-xr-x 1 root root 26 Sep 2 17:20 x86_64-suse-linux alexandru@linux-ampu:~/tar> ls file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 alexandru@linux-ampu:~/tar> ls -l /usr/bin > file1 alexandru@linux-ampu:~/tar> ls -l /tmp > file2 alexandru@linux-ampu:~/tar> ls -l /dev > file3 alexandru@linux-ampu:~/tar> ls -l /root > file4 ls: cannot open directory '/root': Permission denied alexandru@linux-ampu:~/tar> sudo ls -l /root > file4 [sudo] password for root: alexandru@linux-ampu:~/tar> ls -l total 160 -rw-r--r-- 1 alexandru users 603 Sep 2 19:54 file0 -rw-r--r-- 1 alexandru users 132012 Sep 2 19:54 file1 -rw-r--r-- 1 alexandru users 321 Sep 2 19:55 file2 -rw-r--r-- 1 alexandru users 12447 Sep 2 19:55 file3 -rw-r--r-- 1 alexandru users 99 Sep 2 19:55 file4 -rw-r--r-- 1 alexandru users 0 Sep 2 19:53 file5 -rw-r--r-- 1 alexandru users 0 Sep 2 19:53 file6 -rw-r--r-- 1 alexandru users 0 Sep 2 19:53 file7 -rw-r--r-- 1 alexandru users 0 Sep 2 19:53 file8 -rw-r--r-- 1 alexandru users 0 Sep 2 19:53 file9 alexandru@linux-ampu:~/tar> ls -l /usr > file5 alexandru@linux-ampu:~/tar> ls -l /dev > file6 alexandru@linux-ampu:~/tar> ls -l /dev > file7 alexandru@linux-ampu:~/tar> ls -l /dev > file8 alexandru@linux-ampu:~/tar> ls -l /dev > file9 alexandru@linux-ampu:~/tar> ls -lh total 228K -rw-r--r-- 1 alexandru users 603 Sep 2 19:54 file0 -rw-r--r-- 1 alexandru users 129K Sep 2 19:54 file1 -rw-r--r-- 1 alexandru users 321 Sep 2 19:55 file2 -rw-r--r-- 1 alexandru users 13K Sep 2 19:55 file3 -rw-r--r-- 1 alexandru users 99 Sep 2 19:55 file4 -rw-r--r-- 1 alexandru users 603 Sep 2 19:56 file5 -rw-r--r-- 1 alexandru users 13K Sep 2 19:56 file6 -rw-r--r-- 1 alexandru users 13K Sep 2 19:56 file7 -rw-r--r-- 1 alexandru users 13K Sep 2 19:56 file8 -rw-r--r-- 1 alexandru users 13K Sep 2 19:56 file9 alexandru@linux-ampu:~/tar> tar cvzf my1.tar.gz file[0-9] file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 alexandru@linux-ampu:~/tar> ls file0 file2 file4 file6 file8 my1.tar.gz file1 file3 file5 file7 file9 alexandru@linux-ampu:~/tar> tar cvjf my2.tar.bz2 file[0-9] file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 alexandru@linux-ampu:~/tar> tar cvJf my3.tar.xz file[0-9] file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 alexandru@linux-ampu:~/tar> ls -lh total 304K -rw-r--r-- 1 alexandru users 603 Sep 2 19:54 file0 -rw-r--r-- 1 alexandru users 129K Sep 2 19:54 file1 -rw-r--r-- 1 alexandru users 321 Sep 2 19:55 file2 -rw-r--r-- 1 alexandru users 13K Sep 2 19:55 file3 -rw-r--r-- 1 alexandru users 99 Sep 2 19:55 file4 -rw-r--r-- 1 alexandru users 603 Sep 2 19:56 file5 -rw-r--r-- 1 alexandru users 13K Sep 2 19:56 file6 -rw-r--r-- 1 alexandru users 13K Sep 2 19:56 file7 -rw-r--r-- 1 alexandru users 13K Sep 2 19:56 file8 -rw-r--r-- 1 alexandru users 13K Sep 2 19:56 file9 -rw-r--r-- 1 alexandru users 28K Sep 2 20:01 my1.tar.gz -rw-r--r-- 1 alexandru users 22K Sep 2 20:01 my2.tar.bz2 -rw-r--r-- 1 alexandru users 22K Sep 2 20:02 my3.tar.xz alexandru@linux-ampu:~/tar>
- to check the contents of a tar archive, use the command:
tar tvf my1.tar.gz alexandru@linux-ampu:~/tar> tar tvf my1.tar.gz -rw-r--r-- alexandru/users 603 2017-09-02 19:54 file0 -rw-r--r-- alexandru/users 132012 2017-09-02 19:54 file1 -rw-r--r-- alexandru/users 321 2017-09-02 19:55 file2 -rw-r--r-- alexandru/users 12447 2017-09-02 19:55 file3 -rw-r--r-- alexandru/users 99 2017-09-02 19:55 file4 -rw-r--r-- alexandru/users 603 2017-09-02 19:56 file5 -rw-r--r-- alexandru/users 12447 2017-09-02 19:56 file6 -rw-r--r-- alexandru/users 12447 2017-09-02 19:56 file7 -rw-r--r-- alexandru/users 12447 2017-09-02 19:56 file8 -rw-r--r-- alexandru/users 12447 2017-09-02 19:56 file9 alexandru@linux-ampu:~/tar>
- in order to append or update the archive, use the commands from the following output. Please take into consideration that you can not update or append to an compressed archive. You need to uncompress it, make the modifications and then compress it again. Here is an example for the tar.gz compressed archive. We will replace file number 2 with a new one.
gzip -d my1.tar.gz
this will uncompress the archive
tar --delete --file my1.tar file2
this will delete the file2 inside the archive
now create a new file2 with the command
ls -l /usr/bin/ > file2 tar --update --file my1.tar file2
to add the updated file
gzip my1.tar
to compress the archive again.
- you can check the contents with the command:
tar tvf my1.tar.gz
This is the output:
alexandru@linux-ampu:~/tar> ls file0 file2 file4 file6 file8 my1.tar.gz my3.tar.xz file1 file3 file5 file7 file9 my2.tar.bz2 alexandru@linux-ampu:~/tar> gzip -d my1.tar.gz alexandru@linux-ampu:~/tar> ls file0 file2 file4 file6 file8 my1.tar my3.tar.xz file1 file3 file5 file7 file9 my2.tar.bz2 alexandru@linux-ampu:~/tar> tar --delete --file my1.tar file2 alexandru@linux-ampu:~/tar> ls -l /usr/bin/ > file2 alexandru@linux-ampu:~/tar> tar --update --file my1.tar file2 alexandru@linux-ampu:~/tar> gzip my1.tar alexandru@linux-ampu:~/tar> ls file0 file2 file4 file6 file8 my1.tar.gz my3.tar.xz file1 file3 file5 file7 file9 my2.tar.bz2 alexandru@linux-ampu:~/tar> tar tvf my1.tar.gz -rw-r--r-- alexandru/users 603 2017-09-02 19:54 file0 -rw-r--r-- alexandru/users 132012 2017-09-02 19:54 file1 -rw-r--r-- alexandru/users 12447 2017-09-02 19:55 file3 -rw-r--r-- alexandru/users 99 2017-09-02 19:55 file4 -rw-r--r-- alexandru/users 603 2017-09-02 19:56 file5 -rw-r--r-- alexandru/users 12447 2017-09-02 19:56 file6 -rw-r--r-- alexandru/users 12447 2017-09-02 19:56 file7 -rw-r--r-- alexandru/users 12447 2017-09-02 19:56 file8 -rw-r--r-- alexandru/users 12447 2017-09-02 19:56 file9 -rw-r--r-- alexandru/users 132012 2017-09-02 20:15 file2 alexandru@linux-ampu:~/tar>
You should notice the difference in size and creation time on file2 inside the archive.
- to restore the archives, or extract them, use the command:
tar xvzf my1.tar.gz tar xvjf my2.tar.bz2 tar xvJf my3.tar.xz
- to restore the archives with preserved permissions, and to the original user's directory (named restore in my case), use the command:
tar xvzf my1.tar.gz --directory restore --same-permissions tar xvjf my2.tar.bz2 --directory restore --same-permissions tar xvJf my3.tar.xz --directory restore --same-permissions
Here is the output:
alexandru@linux-ampu:~/tar> mkdir restore alexandru@linux-ampu:~/tar> ls file0 file2 file4 file6 file8 my1.tar.gz my3.tar.xz file1 file3 file5 file7 file9 my2.tar.bz2 restore alexandru@linux-ampu:~/tar> tar xvzf my1.tar.gz --directory restore --same-permissions file0 file1 file3 file4 file5 file6 file7 file8 file9 file2 alexandru@linux-ampu:~/tar> ls file0 file2 file4 file6 file8 my1.tar.gz my3.tar.xz file1 file3 file5 file7 file9 my2.tar.bz2 restore alexandru@linux-ampu:~/tar> ls restore/ file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 alexandru@linux-ampu:~/tar> tar xvjf my2.tar.bz2 --directory restore --same-permissions file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 alexandru@linux-ampu:~/tar> tar xvJf my3.tar.xz --directory restore --same-permissions file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 alexandru@linux-ampu:~/tar> ls file0 file2 file4 file6 file8 my1.tar.gz my3.tar.xz file1 file3 file5 file7 file9 my2.tar.bz2 restore alexandru@linux-ampu:~/tar> ls restore/ file0 file1 file2 file3 file4 file5 file6 file7 file8 file9 alexandru@linux-ampu:~/tar> tar xvzf my1.tar.gz --directory new --same-permissions tar: new: Cannot open: No such file or directory tar: Error is not recoverable: exiting now alexandru@linux-ampu:~/tar>
Notice that, at the beginning I created a new directory called "restore" in order to extract the files there. At the end, I intended to extract the files to a directory called "new" that did not exist, and this resulted in a error message and tar command to exit.
Laboratory objective 16: Using tar for backup
Commands used: mkdir, ls, tar, cd, du
Solution:
- create a directory called backup and inside that directory place a compressed tar archive of all the files under /usr/include. Use any of gzip, bzip2 or xzip)
alexandru@linux-vje9:~> mkdir backup alexandru@linux-vje9:~> ls backup Desktop Downloads Music Public Templates VirtualBox VMs bin Documents Dropbox Pictures public_html Videos alexandru@linux-vje9:~> cd backup alexandru@linux-vje9:~/backup> sudo tar zcvf include.tar.gz /usr/include
If you will not use sudo, the following error will be displayed:
alexandru@linux-vje9:~/backup> tar zcvf include.tar.gz /usr/include/ tar (child): include.tar.gz: Cannot open: Permission denied tar (child): Error is not recoverable: exiting now
- notice the efficiency of the compression:
alexandru@linux-vje9:/usr> du -sh /usr/include/ 12M /usr/include/ alexandru@linux-vje9:/backup> ls -l -rw-r--r-- 1 root root 1900325 Sep 2 22:24 include.tar.gz
- list the files in the archive
ls -lh include.tar.gz alexandru@linux-vje9:/backup> tar tvf include.tar.gz
when decompressing, you don't need to give the j, J, or z options, as tar knows what kind of compressed archive it is decompressing.
- create a directory called backup_restore and unpack and decompress the archive there
alexandru@linux-vje9: ~> mkdir backup_restore alexandru@linux-vje9:~> cd backup_restore alexandru@linux-vje9:~/backup_restore> tar xvf ../backup/include.tar.gz alexandru@linux-vje9:~/backup_restore> ls usr alexandru@linux-vje9:~/backup_restore> cd usr/ alexandru@linux-vje9:~/backup_restore/usr> ls include alexandru@linux-vje9:~/backup_restore/usr> cd include/ alexandru@linux-vje9:~/backup_restore/usr/include> ls aio.h complex.h ifaddrs.h nss.h stdio.h aliases.h cpio.h inttypes.h obstack.h stdlib.h alloca.h crypt.h langinfo.h ow-crypt.h string.h a.out.h ctype.h lastlog.h paths.h strings.h arch dirent.h libgen.h poll.h stropts.h arch-v10 dlfcn.h libintl.h printf.h sys arch-v32 elf.h libio.h protocols syscall.h argp.h endian.h limits.h pthread.h sysexits.h argz.h envz.h link.h pty.h syslog.h ar.h err.h linux pwd.h tar.h arpa errno.h locale.h python2.7 termio.h asm error.h malloc.h python3.4m termios.h asm-arc execinfo.h math.h rdma tgmath.h asm-arm fcntl.h mcheck.h re_comp.h thread_db.h asm-arm64 features.h memory.h regex.h time.h asm-c6x fenv.h misc regexp.h ttyent.h asm-generic fmtmsg.h mntent.h resolv uapi asm-hexagon fnmatch.h monetary.h resolv.h uchar.h asm-ia64 fpu_control.h mqueue.h rpc ucontext.h asm-m68k fstab.h mtd rpcsvc ulimit.h asm-metag fts.h net sched.h unistd.h asm-microblaze ftw.h netash scsi ustat.h asm-nios2 gawkapi.h netatalk search.h utime.h asm-openrisc _G_config.h netax25 semaphore.h utmp.h asm-parisc gconv.h netdb.h setjmp.h utmpx.h asm-powerpc getopt.h neteconet sgtty.h values.h asm-s390 glob.h netinet shadow.h video asm-score gnu netipx signal.h wait.h asm-sparc gnu-crypt.h netiucv sound wchar.h asm-tile gnu-versions.h netpacket spawn.h wctype.h asm-x86 grp.h netrom stab.h wordexp.h assert.h gshadow.h netrose stdc-predef.h X11 bits iconv.h nfs stdint.h xen byteswap.h ieee754.h nl_types.h stdio_ext.h xlocale.h
Laboratory objective 17: Working with grep
Grep is a very useful command and in order to be a good Linux system administrator, one needs to know how to work with it. We will use openSUSE as the base distribution for this exercise, but you feel free to use whatever distro you like.
For this exercise, I will use a file containing some text, named "afile". The text inside the file would be as follows:
alexandru@linux-inva:~> cat afile boot book booze machine boots bungie bark aardvark broken$tuff robots
1 a first simple example is to find all words containing a specific string in that file:
alexandru@linux-inva:~> grep "boo" afile boot book booze boots
2 in order to identify the lines on which the strings were found in the file can be obtained with the -n option:
alexandru@linux-inva:~> grep -n "boo" afile 1:boot 2:book 3:booze 5:boots
3 if you would like to print the negative result, as for all other lines that don't contain "boo" you can use the -v option:
alexandru@linux-inva:~> grep -vn "boo" afile 4:machine 6:bungie 7:bark 8:aardvark 9:broken$tuff 10:robots
4 use option -c if you only want to print the number of lines that match the search:
alexandru@linux-inva:~> grep -c "boo" afile 4
5 you can use the -l option that prints only the filenames of files in the search that have lines that match the search string:
alexandru@linux-inva:~> grep -l "boo" * afile grep: backup: Is a directory grep: bin: Is a directory grep: git-test: Is a directory grep: grep-test: Is a directory grep: restore: Is a directory
6 you can also use the -i option for ignoring case:
alexandru@linux-inva:~> grep -i "BOO" afile boot book booze boots
7 you can also use regular expressions in your searches. For example, find lines that end with the letter "e":
alexandru@linux-inva:~> grep "e$" afile booze machine bungie
8 you can also use egrep command, which is similar to using grep -E like this:
alexandru@linux-inva:~> egrep "boots?" afile boot boots alexandru@linux-inva:~> egrep "boot|boots" afile boot boots
9 if you want to search for a special character, like "$" you can not do a simple grep containing the dollar sign, as it is a special character. Thus, the solution is to escape the symbol:
alexandru@linux-inva:~> grep '\$' afile broken$tuff
10 One more useful exercise would be to show information about your user inside /etc/passwd file, by ignoring case:
alexandru@linux-inva:~> grep -i alexandru /etc/passwd alexandru:x:1000:100:alexandru:/home/alexandru:/bin/bash alexandru@linux-inva:~>
11 show the contents of the /etc directory whose names begins with ss:
alexandru@linux-inva:~> ls /etc | grep ^ss ssh ssl
we use ls /etc command, but if we would use ls -l, the result would be different, as using -l, no line would start with "s"...
alexandru@linux-inva:~> ls -l /etc | grep ^ss alexandru@linux-inva:~>
Laboratory objective 18: Working with awk
Awk is in fact a powerful full fledged programming language, but we can use it for special tasks at the command line too, like processing and analyzing text files and data files that are organized by lines and columns. This is very useful when working with lists. For example, lets say that we have a text file called "list" with the following contents:
alexandru@linux-inva:~/grep-test> cat list 1, Fratii Mentzel, Title 545, Price $7.30 2, Joe Cocker, Title 723, Price $7.90 3, Mick Jagger, Title 610, Price $7.90 4, Goran Bregovic, Title 118, Price $7.30 5, Michael Bolton, Title 482, Price $6.50 6, Elvis Presley, Title 335, Price $7.30 7, John Lennon, Title 271, Price $7.90 8, Tina Turner, Title 373, Price $5.50
1 lets say that you would like to see only the contents of the 5th column and print the outcome to a new file called title:
alexandru@linux-inva:~/grep-test> awk '{print $5}' list
545,
723,
610,
118,
482,
335,
271,
373,
alexandru@linux-inva:~/grep-test> awk '{print $5}' list > title
alexandru@linux-inva:~/grep-test> cat title
545,
723,
610,
118,
482,
335,
271,
373,
alexandru@linux-inva:~/grep-test>
2 if you have a column separator that is different then spaces and tabs, you can specify in the awk input that the columns you want to list are separated by commas:
alexandru@linux-inva:~/grep-test> awk -F, '{print $3}' list > title1
alexandru@linux-inva:~/grep-test> cat title1
Title 545
Title 723
Title 610
Title 118
Title 482
Title 335
Title 271
Title 373
alexandru@linux-inva:~/grep-test>
3 you can put a conditional expression before the block (the contents of the {} is considered a block) so that the block will be executed only if the conditional statement is true:
alexandru@linux-inva:~/grep-test> awk '$7=="\$7.30" {print $3}' list
awk: cmd. line:1: warning: escape sequence `\$' treated as plain `$'
Mentzel,
Bregovic,
Presley,
alexandru@linux-inva:~/grep-test>
What it does, is to print all the musician names that have the titles priced at 7.30 dollars.
4 if a line contains a string from a regular expression, the command will print out the element from a specific column of that line. For example, if a line contains the string "30", the command will print out the element from the 3rd column - it has the same result as the previous command:
alexandru@linux-inva:~/grep-test> awk '/30/ {print $3}' list
Mentzel,
Bregovic,
Presley,
alexandru@linux-inva:~/grep-test>
5 if you would like the command to print the entire names, give the following parameters:
alexandru@linux-inva:~/grep-test> awk -F, '/30/ {print $2}' list
Fratii Mentzel
Goran Bregovic
Elvis Presley
alexandru@linux-inva:~/grep-test>
Laboratory objective 19: Working with sed
Sed is a stream editor and it can perform basic text transformations on an input stream, with a high efficiency.
For the purpose of this exercise we will use the same file called "list" as used in the previous exercise, but we will rename it from "list" to "songs".
1 let's make text substitutions to that file, using sed. For example, change the occurrence of "Fratii" to "Tatii" using:
alexandru@linux-inva:~/grep-test> sed 's/Fratii/Tatii/' songs > songs2 alexandru@linux-inva:~/grep-test> cat songs 1, Fratii Mentzel, Title 545, Price $7.30 2, Joe Cocker, Title 723, Price $7.90 3, Mick Jagger, Title 610, Price $7.90 4, Goran Bregovic, Title 118, Price $7.30 5, Michael Bolton, Title 482, Price $6.50 6, Elvis Presley, Title 335, Price $7.30 7, John Lennon, Title 271, Price $7.90 8, Tina Turner, Title 373, Price $5.50 alexandru@linux-inva:~/grep-test> cat songs2 1, Tatii Mentzel, Title 545, Price $7.30 2, Joe Cocker, Title 723, Price $7.90 3, Mick Jagger, Title 610, Price $7.90 4, Goran Bregovic, Title 118, Price $7.30 5, Michael Bolton, Title 482, Price $6.50 6, Elvis Presley, Title 335, Price $7.30 7, John Lennon, Title 271, Price $7.90 8, Tina Turner, Title 373, Price $5.50
2 If you want to change the prices from the list, you can do that too. Let's say you want to change 7.30 to 8.30:
alexandru@linux-inva:~/grep-test> sed 's/7.30/8.30/' songs > songs2 alexandru@linux-inva:~/grep-test> cat songs2 1, Fratii Mentzel, Title 545, Price $8.30 2, Joe Cocker, Title 723, Price $7.90 3, Mick Jagger, Title 610, Price $7.90 4, Goran Bregovic, Title 118, Price $8.30 5, Michael Bolton, Title 482, Price $6.50 6, Elvis Presley, Title 335, Price $8.30 7, John Lennon, Title 271, Price $7.90 8, Tina Turner, Title 373, Price $5.50
3 you can use sed to filter lines in a file or stream. For example, if you only want to see lines containing a certain string, like "Tina" you can use:
alexandru@linux-inva:~/grep-test> sed -n '/Tina/p' songs > tina alexandru@linux-inva:~/grep-test> cat tina 8, Tina Turner, Title 373, Price $5.50
Laboratory objective 20: Working with very large text files
Here is the scenario for the following laboratory: someone who manages a very large e-commerce website which has thousands of user accounts and associated e-mails is asking you to help them solve a problem that they have with some of the user accounts e-mail addresses related to spam and bounces. They don't know anything about their underling operating system or e-mail server. Their exact request is for you to find out where the bounce e-mail addresses are stored, out of all the files related to the addresses use the ones that have the line "Subject: Undelivered Mail Returned to Sender" and further, from those selected, extract the one line that refers to the actual e-mail address and thus has the string "Original-Recipient: rfc822;". Then, create a list of all the e-mails that will actually appear after the "Original-Recipient: rfc822;" string and send the list file back to them to do whatever they want to do with it.
Now, in our exercise, we will not use real e-mail addresses, but some strings of text that we will extract from those files and use them to create a list.
NOTE: you have been given ssh access to the server by the website managers.
1 connect to the server using ssh credentials
ssh -p 2222 username@ipaddress/website
2 first you will need to know what Linux distribution the server is running on
uname -a
The server could be CentOS, Debian, Ubuntu or whatever... RHEL or SLES... We will assume that the running server is Debian
3 you don't know what e-mail server they use, thus you will need to find out using the command:
telnet $HOSTNAME 25
Lets say that this will give you the answer that the e-mail server used is Exim. If you never used Exim before, you will need to find out where the configuration files are and also, where the server stores the log files. Usually, the e-mail files are saved in /home/user/mail where the "user" is the name of the system user. The Exim configuration file is stored under /etc/exim.conf and the log files are stored under /var/log/exim_mainlog and /var/log/exim_rejectlog
4 trying to view the logs will render an "access denied" error, meaning that the user you have been given to connect through ssh does not have permissions to view/modify the files. Running sudo also is not permitted. Thus, all you have to do is to look inside every directory and find out where the wanted files are stored.
5 lets say that you find the files you need under ~/mail/website/no-reply/cur/ and there are thousands of text files that contain the information you need.
6 you will create a list with the filenames of the files that contain the "Subject: Undelivered Mail Returned to Sender" line using the command. I advice you to create a new directory to work with, not to alter the existing one. Thus, you will create a new directory using the command:
mkdir cur_test
inside the ~/mail/website/no-reply/ directory.
grep -rnwl . -e 'Subject: Undelivered Mail Returned to Sender' > ~/mail/website/no-reply/cur_test/file_list.txt
7 now we will copy the files that have the filenames on the list to a new directory:
xargs -a ~/mail/website/no-reply/cur_test/file_list.txt cp -t ~/mail/website/no-reply/cur_test/new_dir/
8 inside the new directory, that now contains all the files you need, we will extract from those files the lines that contain the string "Original-Recipient: rfc822;" and add them into a new file
grep "Original-Recipient: rfc822;" * > ~/mail/website/no-reply/cur_test/mail_list_bounce.txt
9 right now, the newly created file contains, on each single line, the name of the file and strings like the name of the website and "Original-Recipient: rfc822;" and then the actual e-mail address that you need to use. All you need to do now is to delete all the characters on each line before the string "rfc822;" and the before mentioned string. For this we will use the command:
sed -i.bak 's/^.*rfc822//' mail_list_bounce.txt
10 now, all you need to do is to create a unique list, as there are all the instances that any mail address has been used to bounce, but you only need all the e-mails in the list. For this, you can use the command:
sort mail_list_bounce.txt | uniq > unique_list.txt
This is it. You will have a list file containing all the e-mails that you need, or your employer needs.
Laboratory objective 21: Generating random large text files
In this laboratory we will generate some content that we could use for the previous described lab, where we used a large text file. Just if it happens that you don't have a large file, you could simply generate one as I will show you below.
Exercise 1:
Create a text file with 5534 lines, each line containing ten alphanumeric characters.
Solution A:
$ for i in $( seq 1 5534); do < /dev/urandom tr -dc "[a-zA-Z0-9]" | head -c10; echo; done > file1a
This line will execute, for 5534 times, a translation (tr = translate command) of random strings 10 chars long using the /dev/urandom input, then it writes the output to the file "file1a".
Solution B:
$ shuf -n5534 /usr/share/dict/words > file1b
This will generate a file called "file1b" that will contain 5534 random words, each word on one line. It uses the /usr/share/dict/words dictionary words file.
Exercise 2:
We already have a file called "list" that contains 5534 lines of text, and each line has a long string of text (alphanumeric characters). You will have to "add" the contents of the previously created file to the contents of the "list" file, in such way that each word from a line from the file to be added at the end of each line of the "list" file.
Solution:
$ paste list file1a > list2a $ paste list2a file1b > list2b
Exercise 3:
The file "list" used in the previous exercise has the string ".impala.website.welcome.com" on each of its lines, so you will now have to delete it and use the final file using only the strings that are left. Random strings.
mv list2b list3sed -i 's/.impala.websitewelcome.com//g' list3
Here you go, now you have a 5534 line text file containing only random text called "list3" that you can use for the previous laboratory.