Only micro$oft makes it possible…

Cleaning up a micro$oft mess

Part of this post is splitted out from my Command line post (and maybe others), about the newly invented Bash (complete Micro$oft Ubuntu) in windows.

The parts in this post is more related to the mess windows makes on a system drive that has to be cleaned up (for getting back storage, or when it’s time to reinstall windows or take out an old computer of service and replace it with a more recent) when replaced and being reused as a data drive only.

If you did a full clone of the old drive, you can safely just wipe the old once you verified everything is available on the new drive. This is more for those who do a clean installation on a new drive and want to keep user-files on the old drive.

Only Windows makes this mess (OneDrive)

When taking a hard drive containing OneDrive folders out and mounting it as a secondary drive, some files in the OneDrive folders might be “online only” or to be “downloaded on demand”. On that disk, which is disconnected from the OneDrive account, all the files will be there (until disconnected) but not accessible.
The inaccessible files can be detected by the “dir” command, giving the file sizes in parentheses.
One option to clean out the mess made by disconnecting the OneDrive disk is to just delete the whole old OneDrive folder, if you are sure there’s nothing there you want to keep (you files should be stored online).
If you want to be safe, keeping all the files that are available offline in the folder, you need to get rid of the “placeholder” files in there.
This is how I did, it might or might not work for you (always be careful when deleting files).
First, I removed all files in my Pictures (automatic sync from camera) folder with a block size of zero:
stat -c "#rm%brm %n" * | sed 's/#rm0rm/rm/g' >rmscript
./rmscript

I decided it was too much work to use that method for doing all directories recursively inside the OneDrive folder, so for the rest of the online-only files I used another method.
For this, I used the “file” command which gave an error for the inaccessible files “(Invalid argument)”:
find . -type f -exec file {} >>filinfo \;
grep "(Invalid argument)" filinfo >offlinefiles
sed "s;\./;rm './;g" offlinefiles >offlinefiles2
sed "s;: ERROR: ;' #;g" offlinefiles2 >offlinefiles3

What this does is creating a list of all the files starting from the current directory. The next command filters out the lines with the error “(Invalid argument)”. As the names in the list begins with “./”, this is then replaced with “rm ./”, creating commands to remove all the files which was filtered out in the previous command.
I then replace ” ERROR” with a hashtag to tell shell to ignore the rest of the line (“#” starts a comment, if you didn’t know).
Double check the “offlinefiles3” before executing it.

Cleaning up in AppData

For the most of the times, AppData contains only crap which can be deleted without risking loosing anything important, but if you use certain applications, that folder may contain very important data.

%AppData% (your user’s AppData/Roaming) contains settings and data for some applications:
If you have been using Thunderbird as a mail client, you will have all the local databases of email and account settings in %AppData%/Thunderbird.
Firefox (Mozilla/Firefox/Profiles) might also be valuable to save for transferring to another system.
FileZilla (ftp/sftp client) has the settings stored here.
JottaCloud (or branded Jotta-clones like Elgiganten Cloud) store its databases in “Jotta” (no matter if it is Jotta or a branded clone), for more information about the content in the databases, see my post JottaCloud secrets

WSL or Micro$oft Ubuntu

If you have been using WSL (Micro$oft Ubuntu) the default user home location is nested inside AppData/Local (not %AppData%, but you could navigate there then go one step up and enter “Local” from there).
The full location is

AppData/Local/Packages/CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc/LocalState/rootfs

User created files (if you have some) are usually in ‘root’ and in ‘home’,but if you think you have modified files elsewhere (/etc or if you have mysql databases), you can search for them using the oldest known non-system file as a reference timestamp file.
To find out what files you have modified since WSL (if any other than inside ‘home’), run the command ‘ls -lta’ inside your home folder, then take the last (oldest) file in there, and run a second command in the ‘rootfs’ folder:

...1fndgsc/LocalState/rootfs# find . -type f -newer home/youruser/.bash_logout

This will list all files that have been modified since the .bash_logout file (which was my oldest file in /home)

The registry, make things even more complicated

The registry has been the main location for application ans system settings since Windows 95, and will probably be so for a long time into the future.
When you have taken your old system disk offline, and on a new clean installation discover that settings have been lost in all applications there exists some ways to recover contents from your old drive.

In your user’s home directory, usually c:\users\youtoldusername, there is a file called NTUSER.DAT (probably hidden, but you should know how to google to find out how to be able to see and access hidden files). This file is the container for your old user’s settings, a ‘hive’ in HKEY_USERS. This cam be imported using regedit, and at the import time you will be asked to give it a new name (so it won’t overwrite any existing registry content).

One application I use/have used in the past, and assume at least some of you who find my posts is using is PuTTY.
Once you have imported the old registry, you will find the PuTTY sessions in whatever you named your import, followed by:

Software\SimonTatham\PuTTY\Sessions

Click the “Sessions” node,then right click and export.

To import the settings to your user on any other computer, you have to search and replace content in the exported “.reg” file:

HKEY_USERS\whatevernameyouusedatimport\

with

HKEY_CURRENT_USER\

This will overwrite existing values with the content from the import, so it’s up to you to preserve the values you wish to keep (export, to import-overwrite again). If you have not yet started the application (in this case PuTTY) to import settings to, you will have to do that before, otherwise the import would fail (regedit will not create the folder “key” or the full path down to that key if it does not exist, but it will report that the import was successful even when it fails).

Windows update failure (KB5046740) error 0x800f081f

Recently, Windows Update started to fail on one of my newer computers with Win 11 Pro 24H2. Just the specific update named in the title of this part, others following that went through fine. I did all the usual ‘remedies’ for this problem (stop wuauserv etc and remove the cache, multiple restarts, tried manual installation of the update and more), and it just continued to fail.
As a last attempt before reinstalling, I followed “step 5” (since the other things had already been tried, and tried again) in the video (youtube: v=8aYa_B7UgFE). Most of this had already been done, except the ‘dism’ commands:

SC config trustedinstaller start=auto
net stop bits
net stop wuauserv
net stop msiserver
net stop cryptsvc
net stop appidsvc
Ren %Systemroot%\SoftwareDistribution SoftwareDistribution.old
Ren %Systemroot%\System32\catroot2 catroot2.old
regsvr32.exe /s atl.dll
regsvr32.exe /s urlmon.dll
regsvr32.exe /s mshtml.dll
netsh winsock reset
netsh winsock reset proxy
rundll32.exe pnpclean.dll,RunDLL_PnpClean /DRIVERS /MAXCLEAN
dism /Online /Cleanup-image /ScanHealth
dism /Online /Cleanup-image /CheckHealth
dism /Online /Cleanup-image /RestoreHealth
dism /Online /Cleanup-image /StartComponentCleanup
Sfc /ScanNow
net start bits
net start wuauserv
net start msiserver
net start cryptsvc
net start appidsvc

I’m obviously not alone with having this update to fail:
Google search for 'kb5046740%2Bunable%2Bto%2Binstall'

Windows for UNIX users

Command examples written for manipulating files on an old drive with remaining crap from windows.

chown, rm: Delete old windows / program files from second drive

chown -r root:root <Directory>

takeown /F "F:\ProgramData" /A /R /D Y
icacls "F:\ProgramData" /T /grant administrators:F

rm -rf <Directory>
(after chown above)

rd /s /q "F:\ProgramData"

ln
https://www.howtogeek.com/howto/16226/complete-guide-to-symbolic-links-symlinks-on-windows-or-linux/

cp -rp
xcopy <source>\*.* /s/e/f <dest>

Computer games

Game posters

Lucasarts Posters

Sites

Humble Bundle
GOG
GamersGate
Abandonware DOS

Spore

SPORE™ Collection on GoG
Google Search (videos) “Spore GDC 2005”

Spore 35min Demonstration from the GDCe (2005)

The making of spore (FULL)

Will Wright: Spore, birth of a game (TED Talk)

Will Wright and Spore

G4TV’s Icons – Will Wright

Cannon Fodder

OpenFodder
(some free levels + uses data from full Amiga/PC versions)

BeamNG.Drive

Available from Steam and Humble Bundle
https://store.steampowered.com/app/284160/BeamNGdrive/
https://www.humblebundle.com/store/beamngdrive

https://beamng.com/
https://www.beamng.com/threads/your-performance-testing-results.37866/
https://www.beamng.com/threads/performance-statistics-feedback-thread.28306/

Wreckfest

https://www.humblebundle.com/store/wreckfest

Tornado

One of the best flight simulators of its time. Partial source code (the main file ‘flight.exe’) for the simulator was released for enhancement in 2016.
https://github.com/TornadoGame
Game download on myabandonware
https://www.myabandonware.com/game/tornado-41i

Tornado subforum on dogsofwar
http://dogsofwarvu.com/forum/index.php/board,50.0.html
Tornado gameplay by Frankie Kam

Digital Integration Tornado On Twitch (Frankie Kam)


Digital Integration Tornado CD Soundtrack

X-Plane 11

https://www.x-plane.com/
https://www.x-plane.com/email-archives-index/

Captain Joe

Fly with Captain Joe

Command line

Only Windows makes this mess (OneDrive)

This part (about cleaning up the mess OneDrive leaves for online-only files) has been moved to Only micro$oft makes it possible…

Cygwin

Backup windows userfolders

touch backupstamp
for dir in *;
  do ls -ld "$dir";
  tar cf "/cygdrive/s/${dir}.tar" "backupstamp";
  for subdir in "Pictures" "Favorites" "Documents" "Downloads" "Desktop";
    do tar rf "/cygdrive/s/${dir}.tar" "${dir}/${subdir}";
  done;
done

Bash basics

How to output a multiline string in Bash?

expr and if in bash (bash compare with expr calculation) (if -lt compare)

UNIX

The most of the commands below are also valid for Windows 10, since Microsoft invented Ubuntu and bash in 2016.

Mount a partition inside a disk image
Source: https://ktln2.org/2013/10/01/mount-a-partition-inside-a-disk-image/
1: Find the start of the partition you want to mount using parted:

# parted /root/amiga.hdf -s unit b print
Pralloc = 0, Reserved = 2, blocksize = 1, root block at 262176
Model:  (file)
Disk /root/amiga.hdf: 268435456B
Sector size (logical/physical): 512B/512B
Partition Table: amiga
Disk Flags:

Number  Start   End         Size        File system  Name  Flags
 1      32768B  268435455B  268402688B  affs1        DH99

2: set up the loop device, and check it afterwards

# losetup --offset=32768 -f /root/amiga.hdf
# losetup --all
/dev/loop0: [2049]:1443765 (/root/amiga.hdf), offset 32768

3: Mount the partition and check that it is mounted

# mkdir -p /mnt/amiga
# mount /dev/loop0 /mnt/amiga/
# df -h /mnt/amiga
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop0      256M   66K  256M   1% /mnt/amiga

Mounting CD images on Linux

mount /path/to/cdimage.iso /mnt/cdmount -o loop,ro

Mounting .bin / .cue files on Linux
Is not possible, but the .bin and .cue can be converted to .iso as follows using ‘bchunk’ (Binchunker):

sudo apt-get install bchunk
bchunk cdimage.bin cdimage.cue output

Compare the Contents of Two Folders with the Diff Command
https://lifehacker.com/compare-the-contents-of-two-folders-with-the-diff-comma-598872057

rsync
Can I make rsync output only the summary?
summary only in output

awk
Print Column – Change Field Separator – Linux Bash
https://www.shellhacks.com/awk-print-column-change-field-separator-linux-bash/
Remove line break
https://serverfault.com/questions/391360/remove-line-break-using-awk

grep getting confused by filenames with dashes
https://unix.stackexchange.com/questions/364922/grep-getting-confused-by-filenames-with-dashes

HEX dump a file
http://www.theunixschool.com/2011/06/3-different-ways-of-dumping-hex.html

Exclude directory using tar
https://www.linuxquestions.org/questions/linux-software-2/tar-exclude=directory-455908/

Moving and renaming files

Rename files, replace spaces with underscore
https://unix.stackexchange.com/questions/405085/rename-files-to-change-spaces-to-underscore

for file in *.doc *.mp3 *.wav *.txt
do
  mv -- "$file" "${file// /_}"
done

More mass-renaming of files
If you, for example use some program to split a PDF document into single-page files, and that program does not name the files as you want, as in resultig files named as: p_Part_1.pdf, p_Part_2.pdf … p_Part_10.pdf, p_Part_11.pdf … p_Part_100.pdf, p_Part_101.pdf …
The PDFill PDF Editor (tools) does stupid naming like this (you have to supply at least one letter for the file names, and “_Part_” is appended).
Correct naming could be possible by using ‘convert’ from the imagemagick package, but when I tried, the PDFs got large and/or in low quality (converted to images and then back to PDF).

for file in *.pdf; do mv -- "$file" "${file//p_Part_/}"; done
for file in ?.pdf; do mv -- $file 00$file; done
for file in ??.pdf; do mv -- $file 0$file; done

The following search and replace is related to above, but can be used for other purposes:

Search and replace filenames in a text
Add leading zeroes to short numerical filenames (1.pdf = 001.pdf, 10.pdf = 010.pdf)

sed -E 's%/([0-9][0-9]).pdf%/0\1.pdf%' source.1  >source.2
sed -E 's%/([0-9]).pdf%/00\1.pdf%' source.2  >source.1

Another multi-step renaming I did
This time, I renamed the output files from Wondershare Video Converter, in this specific case the DVDs from “The Amiga Years”.
The converted files were in this case named lika:
THE_AMIGA_YEARS_D1_Title_01_02.mp4
THE_AMIGA_YEARS_D1_Title_02_01.mp4
THE_AMIGA_YEARS_D1_Title_03_01.mp4
..
THE_AMIGA_YEARS_D1_Title_03_10.mp4

and
THE_AMIGA_YEARS_D2_Title_02_01.mp4
..
THE_AMIGA_YEARS_D2_Title_02_16.mp4

I also made converted files with the subtitles, and these were named as usual with windows, appending ” (2)” at the end of the filename:
THE_AMIGA_YEARS_D1_Title_03_10 (2).mp4

First step: decide on how you want the files named
I want my file names short, keeping the disc, track and chapter numbers, later appending (manually) the title of the track, so in case for DVD1:
D1.03.02.mp4
D1.03_02 (sub).mp4

When manually adding titles, I will append ” – title”:
D1.03.02 – David Pleasance – Releasing The Amiga CD32 (sub).mp4*
D1.03.02 – David Pleasance – Releasing The Amiga CD32.mp4*

The renaming process:
Be careful when working on your files. Always use a simple echoing command first to determine what will be done.
The common part of the file names for DVD1 is “THE_AMIGA_YEARS_D1_Title_”, and according to my naming goals, that part should be replaced by “D1.”:

for file in THE_AMIGA_YEARS_D1_Title_*.mp4; do echo -- "$file" "${file//THE_AMIGA_YEARS_D1_Title_/D1.}"; done

This will shorten the file names to:
D1.03_10.mp4
D1.03_10 (2).mp4

This next renaming step will replace “(2)” with “(sub)”:

for file in D1.*\(2\).mp4; do mv -- "$file" "${file//(2)/(sub)}"; done

The last step will replace the underscore ‘_’ with ‘.’:

for file in D1.03_*.mp4; do mv -- "$file" "${file//_/.}"; done

Move finished downloads out of the downloads folder
Automatically move finished downloads of specific size (larger than) or any other criteria out to a mounted network drive with more space:

(windows steps)
Mount the share as usual and take note of the drive letter (in the example “S”)

(windows-Ubuntu shell steps)

mkdir /mnt/s
mount -t drvfs S: /mnt/s

The script (edit as suitable) to move the files out of downloads (my script is in the downloads drawer, which is also the cwd):

cd /mnt/c/Users/user-name/Downloads

Find any zip-files, larger than about 20MB, not altered in 30 minutes

#!/bin/sh
find . -type f -mmin +30 -name "*\.zip" -size +20000000c -exec mv {} /mnt/s/downloads/ \;

Run this whenever you want to move files, or automate it as follows:

while true; do date;./move-downloads.sh; sleep 300; done

(Scan for new files to move every 5 minutes)

Finding files

Find directories containing matching files

find . -iname "*.pdf" -exec dirname {} \; | sort | uniq

Find all links to a specific file

find -L . -samefile the-file-with-link-cound-larger-than-1

Find the latest modified file(s) in a directory
https://unix.stackexchange.com/questions/240418/find-latest-files

To print the last 3 accessed files (sorted from the last accessed file to the third last accessed file):

find . -type f -exec stat -c '%X %n' {} \; | sort -nr | awk 'NR==1,NR==3 {print $2}'

To print the last 3 modified files (sorted from the last modified file to the third last modified file):

find . -type f -exec stat -c '%Y %n' {} \; | sort -nr | awk 'NR==1,NR==3 {print $2}'

Find files NOT matching search pattern
Simple.. use -not with the ‘find’ command:

find . -type f -not -name *html

Remove Apache-generated indexes after wget -m
wget web crawler retrieves unwanted index.html index files

find . -type f -name "index.html\?C=?;O=?" -exec rm {} \;

Finding IN files

Use ‘grep’
Finding files which does not match search pattern (use ‘grep -L’, as inverse for ‘grep -l’)

Delete lines in a text file that contain a specific string

Linux only

Mount smb share inside a user home folder

mount -t cifs -o username=theuser,uid=1001,gid=1001 //192.168.1.250/theuser /home/theuser/storage

Where “theuser” is the a) user name on the SMB/CIFS server, b) name of the SMB share, c) the user name on the Linux system it will be mounted on. These do not have to be identical. The given gid and uid is for the Linux user that the share is mounted for. Also, the “nounix” option can be used for SMB servers with buggy unix permission implementations.

Enable /etc/rc.local on Ubuntu 20.04
https://marsown.com/wordpress/how-to-enable-etc-rc-local-with-systemd-on-ubuntu-20-04/

FreeBSD

Periodic – using it to run shell scripts
http://www.freebsddiary.org/periodic.php

4.5. Using the Ports Collection
https://www.freebsd.org/doc/handbook/ports-using.html

Windows 10 (swedish)

Uppdateringshistorik för Windows 10 (1703)

https://support.microsoft.com/sv-se/help/4018124/windows-10-update-history

Uppdateringshistorik för Windows 10 (1607)

https://support.microsoft.com/sv-se/help/4000825

wmic

https://superuser.com/questions/447277/list-all-installed-software-on-pc
https://community.spiceworks.com/how_to/111076-using-wmic-to-retrieve-a-list-of-all-installed-programs
https://social.msdn.microsoft.com/Forums/vstudio/en-US/a9beeed3-99f0-4627-ba1e-aa0161fa58c9/wmic-invalid-xsl-format-or-file-name?forum=windowsgeneraldevelopmentissues

Lista alla installerade program, html
wmic /output:Installed.htm product get /format:”%WINDIR%\system32\wbem\en-US\hform”

Lista alla installerade program, csv
wmic /output:Installed.csv product get /format:”%WINDIR%\system32\wbem\en-US\csv”

Creating a bootable USB stick for Windows installation using diskpart

For the other way around, creating a bootable ISO from a USB:
https://aatayyab.wordpress.com/2016/05/09/convert-bootable-usb-to-an-iso-image/
might also be useful: https://www.dell.com/community/Windows-10/Windows-10-rebootable-iso/td-p/5030000

Cloned from: https://blog.bramp.net/post/2008/06/23/create-bootable-windows-server-2008-usb-stick/

Start diskpart

DISKPART> list disk

     Select the USB device from the list and substitute the disk number below
     when necessary

DISKPART> select disk 1
DISKPART> clean
DISKPART> create partition primary
DISKPART> select partition 1
DISKPART> active
DISKPART> format fs=fat32
DISKPART> assign
DISKPART> exit

xcopy X:\*.* /s/e/f Y:\

     where X:\ is your mounted image or physical DVD and Y:\ is your USB
     device

Diskpart info: http://www.computerhope.com/diskpart.htm
See also: USB install media with WIM file larger than 4GB