Only Windows makes this mess (OneDrive)
This part (about cleaning up the mess OneDrive leaves for online-only files) has been moved to Only micro$oft makes it possible…
Cygwin
Backup windows userfolders
touch backupstamp
for dir in *;
do ls -ld "$dir";
tar cf "/cygdrive/s/${dir}.tar" "backupstamp";
for subdir in "Pictures" "Favorites" "Documents" "Downloads" "Desktop";
do tar rf "/cygdrive/s/${dir}.tar" "${dir}/${subdir}";
done;
done
Bash basics
How to output a multiline string in Bash?
expr and if in bash (bash compare with expr calculation) (if -lt compare)
UNIX
The most of the commands below are also valid for Windows 10, since Microsoft invented Ubuntu and bash in 2016.
Mount a partition inside a disk image
Source: https://ktln2.org/2013/10/01/mount-a-partition-inside-a-disk-image/
1: Find the start of the partition you want to mount using parted:
# parted /root/amiga.hdf -s unit b print
Pralloc = 0, Reserved = 2, blocksize = 1, root block at 262176
Model: (file)
Disk /root/amiga.hdf: 268435456B
Sector size (logical/physical): 512B/512B
Partition Table: amiga
Disk Flags:
Number Start End Size File system Name Flags
1 32768B 268435455B 268402688B affs1 DH99
2: set up the loop device, and check it afterwards
# losetup --offset=32768 -f /root/amiga.hdf
# losetup --all
/dev/loop0: [2049]:1443765 (/root/amiga.hdf), offset 32768
3: Mount the partition and check that it is mounted
# mkdir -p /mnt/amiga
# mount /dev/loop0 /mnt/amiga/
# df -h /mnt/amiga
Filesystem Size Used Avail Use% Mounted on
/dev/loop0 256M 66K 256M 1% /mnt/amiga
Mounting CD images on Linux
mount /path/to/cdimage.iso /mnt/cdmount -o loop,ro
Mounting .bin / .cue files on Linux
Is not possible, but the .bin and .cue can be converted to .iso as follows using ‘bchunk’ (Binchunker):
sudo apt-get install bchunk
bchunk cdimage.bin cdimage.cue output
Compare the Contents of Two Folders with the Diff Command
https://lifehacker.com/compare-the-contents-of-two-folders-with-the-diff-comma-598872057
rsync
Can I make rsync output only the summary?
summary only in output
awk
Print Column – Change Field Separator – Linux Bash
https://www.shellhacks.com/awk-print-column-change-field-separator-linux-bash/
Remove line break
https://serverfault.com/questions/391360/remove-line-break-using-awk
grep getting confused by filenames with dashes
https://unix.stackexchange.com/questions/364922/grep-getting-confused-by-filenames-with-dashes
HEX dump a file
http://www.theunixschool.com/2011/06/3-different-ways-of-dumping-hex.html
Exclude directory using tar
https://www.linuxquestions.org/questions/linux-software-2/tar-exclude=directory-455908/
Moving and renaming files
Rename files, replace spaces with underscore
https://unix.stackexchange.com/questions/405085/rename-files-to-change-spaces-to-underscore
for file in *.doc *.mp3 *.wav *.txt
do
mv -- "$file" "${file// /_}"
done
More mass-renaming of files
If you, for example use some program to split a PDF document into single-page files, and that program does not name the files as you want, as in resultig files named as: p_Part_1.pdf, p_Part_2.pdf … p_Part_10.pdf, p_Part_11.pdf … p_Part_100.pdf, p_Part_101.pdf …
The PDFill PDF Editor (tools) does stupid naming like this (you have to supply at least one letter for the file names, and “_Part_” is appended).
Correct naming could be possible by using ‘convert’ from the imagemagick package, but when I tried, the PDFs got large and/or in low quality (converted to images and then back to PDF).
for file in *.pdf; do mv -- "$file" "${file//p_Part_/}"; done
for file in ?.pdf; do mv -- $file 00$file; done
for file in ??.pdf; do mv -- $file 0$file; done
The following search and replace is related to above, but can be used for other purposes:
Search and replace filenames in a text
Add leading zeroes to short numerical filenames (1.pdf = 001.pdf, 10.pdf = 010.pdf)
sed -E 's%/([0-9][0-9]).pdf%/0\1.pdf%' source.1 >source.2
sed -E 's%/([0-9]).pdf%/00\1.pdf%' source.2 >source.1
Another multi-step renaming I did
This time, I renamed the output files from Wondershare Video Converter, in this specific case the DVDs from “The Amiga Years”.
The converted files were in this case named lika:
THE_AMIGA_YEARS_D1_Title_01_02.mp4
THE_AMIGA_YEARS_D1_Title_02_01.mp4
THE_AMIGA_YEARS_D1_Title_03_01.mp4
..
THE_AMIGA_YEARS_D1_Title_03_10.mp4
and
THE_AMIGA_YEARS_D2_Title_02_01.mp4
..
THE_AMIGA_YEARS_D2_Title_02_16.mp4
I also made converted files with the subtitles, and these were named as usual with windows, appending ” (2)” at the end of the filename:
THE_AMIGA_YEARS_D1_Title_03_10 (2).mp4
First step: decide on how you want the files named
I want my file names short, keeping the disc, track and chapter numbers, later appending (manually) the title of the track, so in case for DVD1:
D1.03.02.mp4
D1.03_02 (sub).mp4
When manually adding titles, I will append ” – title”:
D1.03.02 – David Pleasance – Releasing The Amiga CD32 (sub).mp4*
D1.03.02 – David Pleasance – Releasing The Amiga CD32.mp4*
The renaming process:
Be careful when working on your files. Always use a simple echoing command first to determine what will be done.
The common part of the file names for DVD1 is “THE_AMIGA_YEARS_D1_Title_”, and according to my naming goals, that part should be replaced by “D1.”:
for file in THE_AMIGA_YEARS_D1_Title_*.mp4; do echo -- "$file" "${file//THE_AMIGA_YEARS_D1_Title_/D1.}"; done
This will shorten the file names to:
D1.03_10.mp4
D1.03_10 (2).mp4
This next renaming step will replace “(2)” with “(sub)”:
for file in D1.*\(2\).mp4; do mv -- "$file" "${file//(2)/(sub)}"; done
The last step will replace the underscore ‘_’ with ‘.’:
for file in D1.03_*.mp4; do mv -- "$file" "${file//_/.}"; done
Move finished downloads out of the downloads folder
Automatically move finished downloads of specific size (larger than) or any other criteria out to a mounted network drive with more space:
(windows steps)
Mount the share as usual and take note of the drive letter (in the example “S”)
(windows-Ubuntu shell steps)
mkdir /mnt/s
mount -t drvfs S: /mnt/s
The script (edit as suitable) to move the files out of downloads (my script is in the downloads drawer, which is also the cwd):
cd /mnt/c/Users/user-name/Downloads
Find any zip-files, larger than about 20MB, not altered in 30 minutes
#!/bin/sh
find . -type f -mmin +30 -name "*\.zip" -size +20000000c -exec mv {} /mnt/s/downloads/ \;
Run this whenever you want to move files, or automate it as follows:
while true; do date;./move-downloads.sh; sleep 300; done
(Scan for new files to move every 5 minutes)
Finding files
Find directories containing matching files
find . -iname "*.pdf" -exec dirname {} \; | sort | uniq
Find all links to a specific file
find -L . -samefile the-file-with-link-cound-larger-than-1
Find the latest modified file(s) in a directory
https://unix.stackexchange.com/questions/240418/find-latest-files
To print the last 3 accessed files (sorted from the last accessed file to the third last accessed file):
find . -type f -exec stat -c '%X %n' {} \; | sort -nr | awk 'NR==1,NR==3 {print $2}'
To print the last 3 modified files (sorted from the last modified file to the third last modified file):
find . -type f -exec stat -c '%Y %n' {} \; | sort -nr | awk 'NR==1,NR==3 {print $2}'
Find files NOT matching search pattern
Simple.. use -not with the ‘find’ command:
find . -type f -not -name *html
Remove Apache-generated indexes after wget -m
wget web crawler retrieves unwanted index.html index files
find . -type f -name "index.html\?C=?;O=?" -exec rm {} \;
Finding IN files
Use ‘grep’
Finding files which does not match search pattern (use ‘grep -L’, as inverse for ‘grep -l’)
Delete lines in a text file that contain a specific string
Linux only
Mount smb share inside a user home folder
mount -t cifs -o username=theuser,uid=1001,gid=1001 //192.168.1.250/theuser /home/theuser/storage
Where “theuser” is the a) user name on the SMB/CIFS server, b) name of the SMB share, c) the user name on the Linux system it will be mounted on. These do not have to be identical. The given gid and uid is for the Linux user that the share is mounted for. Also, the “nounix” option can be used for SMB servers with buggy unix permission implementations.
Enable /etc/rc.local on Ubuntu 20.04
https://marsown.com/wordpress/how-to-enable-etc-rc-local-with-systemd-on-ubuntu-20-04/
FreeBSD
Periodic – using it to run shell scripts
http://www.freebsddiary.org/periodic.php
4.5. Using the Ports Collection
https://www.freebsd.org/doc/handbook/ports-using.html