Only micro$oft makes it possible…

Cleaning up a micro$oft mess

Part of this post is splitted out from my Command line post (and maybe others), about the newly invented Bash (complete Micro$oft Ubuntu) in windows.

The parts in this post is more related to the mess windows makes on a system drive that has to be cleaned up (for getting back storage) when replaced and being reused as a data drive only.

If you did a full clone of the old drive, you can safely just wipe the old once you verified everything is available on the new drive. This is more for those who do a clean installation on a new drive and want to keep user-files on the old drive.

Only Windows makes this mess (OneDrive)

When taking a hard drive containing OneDrive folders out and mounting it as a secondary drive, some files in the OneDrive folders might be “online only” or to be “downloaded on demand”. On that disk, which is disconnected from the OneDrive account, all the files will be there (until disconnected) but not accessible.
The inaccessible files can be detected by the “dir” command, giving the file sizes in parentheses.
One option to clean out the mess made by disconnecting the OneDrive disk is to just delete the whole old OneDrive folder, if you are sure there’s nothing there you want to keep (you files should be stored online).
If you want to be safe, keeping all the files that are available offline in the folder, you need to get rid of the “placeholder” files in there.
This is how I did, it might or might not work for you (always be careful when deleting files).
First, I removed all files in my Pictures (automatic sync from camera) folder with a block size of zero:
stat -c "#rm%brm %n" * | sed 's/#rm0rm/rm/g' >rmscript
./rmscript

I decided it was too much work to use that method for doing all directories recursively inside the OneDrive folder, so for the rest of the online-only files I used another method.
For this, I used the “file” command which gave an error for the inaccessible files “(Invalid argument)”:
find . -type f -exec file {} >>filinfo \;
grep "(Invalid argument)" filinfo >offlinefiles
sed "s;\./;rm './;g" offlinefiles >offlinefiles2
sed "s;: ERROR: ;' #;g" offlinefiles2 >offlinefiles3

What this does is creating a list of all the files starting from the current directory. The next command filters out the lines with the error “(Invalid argument)”. As the names in the list begins with “./”, this is then replaced with “rm ./”, creating commands to remove all the files which was filtered out in the previous command.
I then replace ” ERROR” with a hashtag to tell shell to ignore the rest of the line (“#” starts a comment, if you didn’t know).
Double check the “offlinefiles3” before executing it.

Cleaning up in AppData

For the most of the times, AppData contains only crap which can be deleted without risking loosing anything important, but if you use certain applications, that folder may contain very important data.

%AppData% (your user’s AppData/Roaming) contains settings and data for some applications:
If you have been using Thunderbird as a mail client, you will have all the local databases of email and account settings in %AppData%/Thunderbird.
Firefox (Mozilla/Firefox/Profiles) might also be valuable to save for transferring to another system.
FileZilla (ftp/sftp client) has the settings stored here.
JottaCloud (or branded Jotta-clones like Elgiganten Cloud) store its databases in “Jotta” (no matter if it is Jotta or a branded clone), for more information about the content in the databases, see my post JottaCloud secrets

WSL or Micro$oft Ubuntu

If you have been using WSL (Micro$oft Ubuntu) the default user home location is nested inside AppData/Local (not %AppData%, but you could navigate there then go one step up and enter “Local” from there).
The full location is

AppData/Local/Packages/CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc/LocalState/rootfs

User created files (if you have some) are usually in ‘root’ and in ‘home’,but if you think you have modified files elsewhere (/etc or if you have mysql databases), you can search for them using the oldest known non-system file as a reference timestamp file.
To find out what files you have modified since WSL (if any other than inside ‘home’), run the command ‘ls -lta’ inside your home folder, then take the last (oldest) file in there, and run a second command in the ‘rootfs’ folder:

...1fndgsc/LocalState/rootfs# find . -type f -newer home/youruser/.bash_logout

This will list all files that have been modified since the .bash_logout file (which was my oldest file in /home)

OpenWrt configuration notes

This post is a continuation of OpenWrt on Raspberry Pi 4 (and CM4)

It’s more about OpenWrt than related to Raspberry Pi hardware like the Pi4 and CM4.

Securing the WAN interface

In the first part, I gave instructions on how to open for SSH and LuCI web access on the WAN interface “the easy way”. This is in no way recommended when the router is moved over to an external WAN on the eth0 interface.
These ports (22 and 80) will be targets for network scanners on the outside, which will find your external IP on the router in a very short time.
Also, if you want to redirect traffic from the outside to these ports on the router, I suggest you use any other port numbers than well-known service ports as 22, 80, 8080, 8000, 443, 8443, 20-21 and 25. There are a lot more recommended to avoid redirecting to the router, as explained in Huawei’s tech note
High-Risk Ports: What Are the Common High-Risk Ports and How to Block Them

Make it safer
If you decide you want to be able to access and configure the router from the outside, your first step should be to set a secure password for the ‘root’ user. The next step is to use other ports than the default well-known ones for SSH (22) and HTTP (80).
This can be done both from LuCI and by editing the firewall configuration file (/etc/config/firewall) directly. I will show and explain how to do it directly in the file (by accessing the router through SSH), as it’s easier to explain, and finding out how to do it in LuCI can be done later (just check the tabs in the firewall configuration there to see how it was done).
Selected ports for my example:
SSH to OpenWRT router from outside, port 16322
HTTP to OpenWRT management page from outside, port 16380

Pretty self-explanatory in the configuration file:

config redirect
        option name 'OpenWRT LuCI'
        option target 'DNAT'
        list proto 'tcp'
        option family 'ipv4'
        option src 'wan'
        option src_dport '16380'
        option dest_port '80'

config redirect
        option name 'OpenWRT SSH'
        option target 'DNAT'
        list proto 'tcp'
        option family 'ipv4'
        option src 'wan'
        option src_dport '16322'
        option dest_port '22'

Reload the firewall configuration by using the command ‘/etc/init.d/firewall reload’
In LuCI, the above will look like this (I modified my config to match what was entered into the file, therefor the notice about unsaved changes)

This is much better than using the well-known ports, but still the ports could be scanned for on the WAN address.
To secure it a bit more, it’s possible to limit to allow access only from known IP-addresses or networks.
To limit to only one IP-address, it can be done by to the relevant redirect section inserting

        option src_ip '8.8.8.8'

This will only allow access to the port from the specified IP (which in this case is one of Google’s DNS servers, which is very unlikely to happen).
In LuCI, the changes above will be visible under “Advanced settings” of the port forward rule.

Using IP sets for the rules
IP sets can be used to allow or deny traffic from multiple IP addresses without having to have a separate rule for each block/allow.
In case for a simple IP-address list, the configuration section would look like this in the /etc/config/firewall file:

config ipset
        option name 'Trusted'
        option comment 'Trusted networks'
        option family 'ipv4'
        list match 'src_ip'
        list entry '8.8.8.8'
        list entry '4.0.0.0/9'

Replace the ‘option src_ip’ in the redirect rule with

        option ipset 'Trusted'

This is the way it will be done if configured using LuCI. A more effieient way is to use include files with hashed IP sets. This is half-explained in a post in the OpenWRT forum: Port forward using ipset
The solution here is to recreate the IP set every time it will be used, but it should be possible to store the IP set and include it in the configuration.

For more information about IP sets, see the excellent OpenWRT documentation: IP set examples

I will have to investigate that in more depth when blocking known bad-bahaving IP hosts or networks.

Redirecting traffic from WAN to a device on the LAN
This is done more or less in the same way as above, but also adding ‘option dest_ip’ to the redirect section:

config redirect
        option name 'SSH to linux server'
        option target 'DNAT'
        list proto 'tcp'
        option src 'wan'
        option src_dport '16222'
        option dest 'lan'
        option dest_ip '172.16.2.11'
        option dest_port '22'
        option ipset 'Trusted'

You can use whatever port you want on the outside (16422 in the example) and any IP on the inside which the router can reach (the example needs a larger netmask: my examples assumes that the router lives on 172.16.3.254, so for it to reach hosts at 172.16.2.x, the netmask have to be ‘255.255.254.0’ (/23) or wider).

If you want to redirect any of those well-known ports to anything inside the LAN, as the ‘Linux server’ above (‘ipset’ limitation removed here, but if you want to keep the server private, just leave it in):

config redirect
        option name 'SSH to linux server'
        option target 'DNAT'
        list proto 'tcp'
        option src 'wan'
        option src_dport '80'
        option dest 'lan'
        option dest_ip '172.16.2.11'
        option dest_port '80'

To be continued (in this post or in another). This is set to be published at the end of 3 Dec 2023 instead of keeping it secret until I decide to release it.

Notes on DHCP

DHCP scope on LAN
As I mentioned in the previous part, the start address of the LAN DHCP scope is configured as an offset from the router’s network address. So unless you have the LAN interface on a single C-subnet, you might have to adjust the start of the scope accordingly. I’m not going into exactly what to set the start value to, you either know it, or you can test and read the results in the logs.

Just a simple example: If you want to have the DHCP scope on a C-net upwards from the start of the LAN segment, just add 256 to the offset; router = 172.16.3.254, netmask = 255.255.254.0, network = 172.16.2.0. To set DHCP range to 192.16.3.100–250, you will have to set the ‘start’ to 356 and the ‘limit’ to 150.

Save and apply changes
It the changes are made from LuCI, ‘save & apply’ will do whatever is needed for the changes to take effect. If you do changes by editing the configuration file (/etc/config/dhcp) directly, you will need to reload the configuration for dnsmasq:

/etc/init.d/dnsmasq reload

Static leases
Static leses are configured in /etc/config/dhcp (which is the file changed by LuCI). A section like this (depending on what you configure through LuCI) is added to that file:

config host
        option name 'Chromecast'
        option mac '90:ca:fa:77:88:99'
        option ip '172.16.3.72'
        option leasetime 'infinite'

Simple enough that requires no explanation, except that the statically assigned IP address is allowed to be outside the configured DHCP range (the address is a bit below the scope here). Use static leases for stuff you want to know which IP addresses they get, without having to make a static configuration on the device itself.

Dropping an active DHCP lease
Active leases are stored in /tmp/dhcp.leases
Remove the corresponding line of the lease you want to drop, then restart udhcpc:

PID=`pidof udhcpc` && kill -SIGUSR1 $PID

Changing display name of known devices
Edit the file /etc/ethers to change display names of detected devices on DHCP. Restart dnsmasq afterwards to get the new names displayed in LuCI.

Online resources

IP subnet calculator
https://www.calculator.net/ip-subnet-calculator.html

MAC address vendor lookup
https://mac.lc/
https://hwaddress.com/

Amiga Productivity Software

Cloned from various wiki pages

Introduction

This article describes productivity software written for the Amiga series of computers, and describes the AmigaOS operating system and its derivatives AROS and MorphOS. This is the main article, Amiga Software Split. See also the related articles “Amiga Internet and Communications Software”, “Amiga Music Software”, “Amiga Programming Language”, and “Amiga Support and Maintenance Software” for additional information about software running on Amigas.

History

The Amiga initially supported well-known software titles such as WordPerfect, Electronic Arts’ Deluxe Paint, and Lattice C. One of the first all-in-one graphics and video editing packages, Newtek’s Video Toaster, started on the Amiga. The Video Toaster was one of the few accessories for the “big box” Amigas (2000, 3000, and 4000) that used video slots, allowing a user to turn his Amiga into the center of an entire television production suite. Newtek’s later addition of Video Flyer enabled the first non-linear video editing program for the Amiga. Amiga brought 3D ray-traced graphics to the public with Sculpt 3D. Before the Amiga, raytracing was only available on dedicated graphics workstations such as SGI. Other ray tracing software also included TurboSilver. The Amiga is well known for its 3D rendering capabilities, and many titles have been added to the mix over the years. Some titles, such as Maxon’s rendering software Cinema 4D and his Newtek’s LightWave, which was originally part of Video Toaster, were later ported to Microsoft Windows and continue to thrive there. The Video Toaster itself has also been ported to the Windows platform. LightWave was used for low-cost computer-generated special effects in the early 1990s, and Babylon 5 is a notable example of a TV series that utilized LightWave. Even Microsoft was creating software for use on the Amiga. AmigaBASIC is an advanced BASIC software development environment with integrated development environment (IDE) created by Microsoft under contract.

Graphics software

The Amiga started in 1985 with a stronger attitude toward graphics than other PCs of the era, due to its unique hardware and multimedia chipset. The graphics chip Agnus has bit blitter and copper circuits that can access RAM directly, steer it with DMA (direct memory access) privileges, move pixel ranges on the screen, and directly process the TV’s electron beam. I was there. It can render graphic screens with different color depths (his HAM mode with 2, 4, 8, 16, 32, 64 and 4096 colors) from 320×200 up to 720×576 pixel graphic pages. Amiga includes Graphicraft, Deluxe Paint, TVPaint, Photon Paint, Brilliance! (a program realized entirely based on the suggestions and wishes of renowned computer artist Jim Sachs), Aegis Images, ArtEffect, fxPAINT, and a huge number of other applications. Released graphics software program. By IOSpirit, Cloanto’s Personal Paint, Photogenics, Express Paint, Digi Paint, XiPaint, PerfectPaint, SketchBlock 24-bit paint programs (by Andy Broad, for AmigaOS 4.x users)

Graphic applications on AmigaOne systems

Unlike the Commodore Amiga system, the AmigaOne system does not have an integrated multimedia chipset. AmigaOne systems, like Macs and PCs, feature a sports AGP/PCIe graphics card, built-in audio AC’97 sound system, and even some professional models can use PCI/PCIe audio cards. Faster CPU performance enhancements and the availability of standard enhanced graphics cards create a new generation of graphics software for AmigaOne machines such as the Hollywood “Visual Programming Suite”. This also makes it easier to port modern open source software like Blender3D.

Visual programming

The Hollywood Suite program by German software house Airsoft SoftWair is a multimedia and presentation program available for all Amigas (AmigaOS, MorphOS, AROS) and as recently as 2010 a version of Hollywood was also made available for Microsoft Windows. became. You can load Scala projects and Microsoft PowerPoint “.PPT” files. Its module Hollywood Designer is not only a modern multimedia authoring software, but also a truly complete cross-platform multimedia application layer that allows you to create entire Amiga programs through a visual design approach. It can also save executables in a variety of formats, including 68k Amiga, WarpUP, AmigaOS 4, MorphOS executables and his Intel X86 code for AROS. The latest version of Hollywood allows you to create executable programs for Intel Windows machines, Mac OS X for PPC processors, and Mac OS X for Intel processors.

Modern graphic software

There is some fairly modern graphics software available for AmigaOne machines, and some is still available for the Amiga platform. Born in 1991, TV Paint was one of the first commercial 32-bit graphics software on the market. The latest version of the Amiga (3.59) was released in 1994 and is indeed publicly distributed, although the source code is still proprietary. It’s still a viable graphics program and continues to be used despite its age due to its ease of use and sheer number of features. Programs like Candy Factory for AOS 4.0 are designed to create special effects for images, brushes, and fonts to create gorgeous Internet objects and buttons used in web page design. Pixel Image Editor (formerly Pixel32) is available for MorphOS. Blender 3D is one of the best open source cross-platform software. Also, the first pre-release of GIMP is available for He AmigaOS 4.0 through the AmiCygnix X11 graphics engine. Since release 2.1 in 2008, MorphOS includes its own standard painting utility called Sketch, which is simple but powerful. AROS bundles the last free version of Luna Paint, which is actually a commercial paint program for various operating systems.

Graphic utilities

As with any operating system, there cannot be only bitmap and vector painting software. Aimed at allowing authors to directly create drawings or manipulate existing image files, these leading software are surrounded by unique features created to support the leading graphics programs. There is a vast market for graphic utilities that For example, the Amiga has very professional software, including at least some of the most important and widely used software on the Amiga market, such as Cinematte, CineMorph, Morph Plus, Impact!, Essence, Magic Lantern, and Pixel 3D Pro. It is worth noting to mention a few. Of the vast array of graphics utilities that were available for purchase by seasoned and professional users of the Amiga platform during the Golden Age, here are just a few of the most notorious. Cinematte utilities make it easy to create complex photorealistic composites of subjects shot against blue-screen or green-screen backgrounds. Accurate blue screen compositing uses the same advanced technology used in film technology around the world. CineMorph is a program that automatically creates a morphing effect between two specified original images and a composite his third image, or any animated movie associated with the morphing effect. Morph Plus performed the same effect as Cinemorph. Impact! I created a physics simulation in a 3D scene. Essence was a texture maker that applied textures to his 3D surfaces of objects created with 3D tracing programs. Amiga Magic Lantern is a true color animation compressor and player for the Amiga that uses the Pixel 3D Pro utility to create models of 3D objects and save them in various 3D formats, or to convert model objects into 3D files. It was used to convert from one format to another.

Vector graphics

Common formats widely used for Amiga vector graphics are EPS and IFF DR2D. This is because the Amiga was the first platform to run Ghostscript natively, and IFF DR2D is the original vector graphics produced by the Amiga ProVector and later adopted by other applications such as Art Expression and Professional Draw. This comes from the fact that it is standard. The most popular Amiga drawing and vector graphics utilities are Aegis Draw, his ProDraw (Professional Draw) from Gold Disk Inc., DrawStudio, Art Expression, and ProVector. You can also use Professional Page and PageStream tools for some basic vector graphics. The newest vector graphics programs on the Amiga are actually MindSpace 1.1, primarily aimed at designing flowcharts, mindtables, UML, and diagrams, and Steam Draw, a 2D simple vector painting program available on MorphOS .

Flash and SWF

SWFTools is a collection of command-line programs for converting and saving various raster (bitmap) image formats to the Flash SWF vector animation format.

Tracing software

AmigaOS provides Autotrace, Potrace, and XTrace, widely used, free, and distributable vector-to-graphics conversion capabilities that also run on AROS Amiga open source clone OS and MorphOS Amiga-Like systems. The Desktop Publishing software PageStream includes a tracing utility as bundled software. The structured drawing program ProVector had an optional add-on tracing utility named StylusTracer.

DXF, EMF, SVG file formats

Various programs can read DXF (almost all Amiga CAD programs), EMF, SVG, CGM, GEM and WMF. An example of a conversion tool that reads many formats and outputs DR2D is the Amiga program MetaView. There is also an SVG data type that provides direct OS support for loading and saving files in SVG (Scalable Vector Graphics) format on any program.

Computer aided design

The Amiga was initially thought to offer the most powerful graphics platform at an affordable price.[1][2] A variety of CAD programs were available, including X-CAD, IntelliCAD, DynaCaDD, MaxonCAD, IntroCAD, and even programs for designing and testing electronic circuits such as ElektroCAD.

Animation, Comics and Cartoons

The Amiga’s unique capabilities in multimedia and the capabilities of its bit-blitter circuitry allowed it to perform advanced animation and video authoring at a professional level in the 1980s, and led to the creation of a vast amount of software that also catered to this field. I did. Top of the professional video editing market. The Amiga supports animation programs such as Aegis Animator, Lights!Camera!Action!, DeLuxe Video, Disney Animation Studio, Deluxe Paint versions 3 and later, The Director (a BASIC-like language for animation), Scala, and Vision. It was possible. From Commodore itself, VisualFX from ClassX, Adorage multi-effects program from proDAD and Millennium from Nova Design. ImageFX, art club professional. Amiga had various animation software. Comic Setter for creating printed comics by placing brushes representing cartoon characters, combining them with background images, and overlaying appropriate frames and “speech bubbles” with their own text speech and captions. It was an interesting tool. You can also print your comics in color. Disney Animation Studios was one of the most powerful 2D programs to bring animation to life. Released on the Amiga, the program also features a full cell-frame preview, and was used by many animation studios around the world during that era, and is still used today by several European studios as a useful preview tool. It has been. This software is primarily used by independent and animation studios. Amateur animator.

Authoring and VideoFX

The golden age of the Amiga had access to a huge range of animation and video authoring software. Aegis Animator, Lights!Camera!Action!, DeLuxe Video, Disney Animation Studio, Deluxe Paint 3 and later versions, The Director (a BASIC-like language for animation), Scala, Commodore’s own Amiga Vision, ClassX’s VisualFX, proDAD’s Adorage Multi Effect program, Andreas Maschke’s Wildfire (later ported to Java by the author), Nova Design’s Millennium, Nova Design’s ImageFX, and ArtDepartment Pro.

3D modeling, rendering and animation

3D rendering and animation software includes Sculpt 3D, TurboSilver, Aladdin4D, Videoscape 3D, Caligari, Maxon Cinema4D, Imagine, Newtek’s LightWave, Realsoft’s Real3D, Vista Pro, World Construction Set 3D terrain rendering programs, and Italian company Eyelight’s Includes Tornado3D.

Amateur and professional video editing

The Amiga was the first commercial Amiga to enable video editing by amateurs and professionals, with connection to TV sets, video codecs, chroma keying, processing of genlock signals, full-screen capability with overscan functionality, and superior noise. A computer platform. -Gain ratio. The Amiga and its video peripherals (primarily genlock boxes and digitization boxes) were available at affordable prices in the 90s, and this made the Amiga one of the leading platforms in the professional video market. It could also handle broadcast video production (Newtek VideoToaster), and despite the demise of the Commodore, the Amiga saw its golden age as a professional video platform around 1992-1994. There was a huge amount of video platforms available for the Amiga. Resale of GFX and image gallery data files applicable to all kinds of video software, graphics equipment, and video production. Among these software is the leading Amiga for desktop video with both linear and non-linear editing with 4.2.2 features, called NLE, made by Newtek and available in the VideoToaster Flyer external module for video toasters. It’s worth mentioning video editing programs. (non-linear editing), Amiga MainActor, Broadcaster 32 and Elite (with Producer software), Wildfire for vfx by Andreas Maschke (now Java), extended Amiga card PAR, VLab Motion (with Movishop software), and VLab Pro.

Word processing and page layout

Desktop video proved to be the main market for the Amiga, but starting with Textcraft, the first Amiga text program that combined a real word processor with advanced text, a proliferation of word processing, page layout, and graphics software became professional. met the needs of Although it is an editor, you can change the page layout, fonts, increase or decrease the width, change colors, and add colored images to the text. Notable word processing programs for the Amiga included the then industry standard WordPerfect up to version 4.1, Shakespeare, Excellence, Maxon Word, Final Writer, Amiga Writer, Scribble!, ProWrite, Wordworth, and a small version of Personal Write by Cloanto. will appear. Page layout software included Page Setter and Professional Page from Gold Disk Inc. and PageStream from Soft-Logik, now known as Grasshopper LLC. Only PageStream has been ported to other platforms and continues to be developed and supported by developers. Graphics software includes vector drawing applications such as Art Expression from Soft-Logik, ProVector from Stylus, Inc. (formerly Taliesin), Draw Studio, and Professional Draw from Gold Disk Inc. The Amiga didn’t have what we would call an office suite, but it did have integrated software. Pen Pal was a word processor with an integrated database and form editor. graffiti! ,analysis! And get organized! We have summarized them as Works! A suite that combines word processing, spreadsheets, and databases. Although the names were similar, it had nothing to do with Microsoft Works. The page layout language LaTeX was available in two ports of his, AmigaTeX, which is no longer available (the first his LaTeX can be edited with a front-end program), and PasTEX, available in the Aminet repository. The latest software AbiWord is now available for AmigaOS 4.0 via the AmiCygnix X11 graphical engine, and Scriba and Papyrus Office pre-releases are available for MorphOS.

Text editors

Text editors available for the Amiga include Vim, Emacs, MicroEMACS (included), Cygnus Editor, also known as CED, and GoldED, which evolved into the Cubic IDE in 2006. UNIX ne editor and vi clone Vim was originally developed on the Amiga. Development of text editors didn’t stop with the Amiga. Since 2001, MorphOS has had a limited edition of GoldEd available called MorphEd, and since 2008 Cinnamon Writer and NoWin ED, a universal editor that works on platforms like the Amiga. Cinnamon Writer adds new features with every new release and aims to be a full-featured word processor.

Database and spreadsheets

In the first days of the Amiga (1986-1989), cross-platform spreadsheets were available, such as MaxiPlan for MS-DOS and Macintosh. Logistix (real name LoGisTiX) was one of the first spreadsheets for the Amiga, the Microfiche Filer Plus, a database that gave users a data exploration experience similar to using microfilm. SuperBase was one of the best programs available for C64. It was later ported to Atari, Amiga, and then PC. On the Amiga, however, it became the standard reference, available in his two versions, Superbase Personal and SuperBase Professional. It could handle SQL databases and had a query internal language such as BASIC. For many years before Microsoft Access, you could create forms and masks on records and incorporate multimedia files into records. Superbase also had VCR control style buttons for browsing records in the database. Softwood File II was another simple multimedia database that later evolved into Final Data. This is a great database available for Amiga by Softwood Inc. From the same company was the very powerful spreadsheet Final Calc, similar to TurboCalc from the German company Schatztruhe. ProChart is a tool for drawing flowcharts and diagrams. Analyze! It was a fairly versatile (for its time) spreadsheet developed for the Amiga. sort out! It was a flat file database package. Gnumeric spreadsheets have also been ported to the Amiga through an X11 engine called AmiCygnix. More recently, MUIbase was born, and his MySQL database language, which is primarily cross-platform, is also referenced on his Amiga. SQLite is a self-contained, embeddable, zero-configuration SQL database engine, also available on AmigaOS 4 and MorphOS. In February 2010, Italian programmer Andrea Palmatè ported his IODBC standard to his AmigaOS 4.

Science, entertainment and special use programs

Maple V is one of the best general-purpose mathematical software (also known as Mathematic-CAD) ever created. It was also available on the Amiga and was highly appreciated by many scientists who were using Amiga at the time. Distant Suns, Galileo, Digital Almanac, and Amiga Digital Universe (by Bill Eaves for OS4) were stargazing programs and astronomical calculators. In the era of CDTV, many history, science, and science timetables, including Science Timetable, Innovation, Business Timetable, Politics, Grolier Encyclopedia, Guinness Disc, Video Creator, Dictionary of American Heritage, Illustrated Bible, Illustrated Works of Shakespeare, etc. , an art CD was sold. etc. were available.

Entertainment

For the Amiga, there were literally hundreds of entertainment software. Notable programs for children and learning include Free Spirit Software’s Adventures in Math series of floppy disks, Unicorn Software’s Animal Kingdom series of disks, Barney Bear Software’s complete Art School series, the Discovery series including Discovery trivia, Donald’s And so on. Alphabet Chase, Disney Software’s Mickey’s 123’s and Mickey’s ABC, the Electric Crayon and Ferngully series of educational coloring book software (Ferngully was taken from an animated film), the Fun School series of discs, the famous Broderbund Kid Pix disc set from the 90s The Miracle Piano Teaching System to Teach Children Music, The Many Stories of Mother Goose, Centaur Software’s World Atlas.

Fractals, virtual reality, artificial intelligence

Elena Novaretti’s ZoneXplorer is considered by Amiga users to be one of the best fractal experience programs ever created on the Amiga, if not on the platform. In 1989, he created Haitex Resources’ X-Specs 3D glasses, one of the first interactive 3D solutions for home computers. Vivid Group Inc.’s multimedia interactive TV non-immersive virtual reality exploration software Mandala and W-Industries’ (then Virtuality Inc.) Virtuality System Virtuality 1000 CS 3D VRML all-immersive simulator were also created on the Amiga and were used for gaming entertainment. was used for Based on the A3000 and used in large arcade facilities and theme parks. Magellan v.1.1 (artificial intelligence software) is not to be confused with Directory Opus Magellan. It was a program that emulated the responses of artificial intelligence on the Amiga by creating heuristically programmed rules based on machine learning in the form of supervised learning. The user selects the AI ​​decision tree and decision table system that the Magellan program features, inputs the object and desired output, and specifies all relevant conditions and conditions that the machine must follow to output a given pseudo-intelligent solution. Write the rules. problem.

Route planning

AmiATLAS v.6 was a complete route planner tool for Amiga computers. We provide interactive maps around the world and find the best routes to get from one place to another. It also featured multiple map loads, an integrated CityGuide system with information on interesting towns, places, and regions (some with photos as well), and information on many parks and points of interest. .

Personal organizer, notebook, diary software

Digita Organizer v.1.1 by Digita International is the best Amiga program that allows users to note dates, meetings, and remember expiration dates. PolyOrga for MorphOS, by Frédéric Rignault.

Personal budget, home banking, accounts

Easy Banker、Home Account、Small Business Account、Small Business Manager、Account Master、Accountant、AmigaMoney、Banca Base III、HomeBank、CashMaster、Counting House など。

Software for special purposes

AVT (Amiga Video Transceiver) was originally developed by “Black Belt Systems” (USA) around 1990 for the Amiga home computers that were popular all over the world before the IBM PC family achieved sufficient audio quality. Software and hardware slow scan television system. With the help of a special sound card. Richmond Sound Design (RSD) created both Show Control (also known as MSC or “MIDI Show Control”) and theater sound design software. These software were widely used in the theater, theme parks, exhibits, exhibitions, stage management, shows and themed entertainment industry. In the 1980s and his 1990s, and at some point in the mid-90s, many high-profile shows at major theme parks around the world were controlled by Amigas simply through software called Stage Manager. It has since evolved into a Microsoft Windows version called ShowMan. Walt Disney World alone hosts dozens of shows, as well as other Disney, Universal Studios, Six Flags, Madame Tussauds properties, as well as the Mirage Hotel Volcano, Siegfried and Roy shows, and the MGM Grand EFX. It includes hundreds of theaters, including shows, Broadway theaters, London’s West End, many of the Royal Shakespeare Company’s theaters, most of Branson, Missouri’s theaters, and many theaters on cruise ships. RSD bought used Amigas on the web, reconditioned them to provide enough systems for all the shows specified, and only stopped offering new Amiga equipment in 2000. The number of shows on cruise ships and themed venues remains unclear, but is operated by Amigas. .

References

^ An interesting interview with Italian manga studio Strane Mani is available on Amiworld.it (in Italian). ^ Information about the virtuality of the Amiga Hardware site

Videos
























Apache HTTPd and PHP security

Notes on setting up Apache and PHP to run under user account

Requirements: base installation with PHP using FPM for Apache

How things are connected

The pool configuration file is the file controlling the ‘server’ (proxy) for PHP. The ‘listen’ line in this file is telling where proxy requests should be accepted.
‘user’ and ‘group’ tells under which user account the process should be run. ‘listen.owner’, ‘listen.group’ and ‘listen.mode’ can be set to limit access to the proxy (by other users/sites).

Sample PHP FPM pool configuration
For each version of PHP that should be available, create a pool configuration file (in ‘/etc/php/<version>/fpm/pool.d/’) like:

[8001]
user = $pool
group = $pool

listen = /run/php/php8.2-fpm-$pool.sock

listen.owner = $pool
listen.group = $pool

pm = dynamic
pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.status_path = /fpm-status

(change the “listen =” line, so it matches the PHP version you wish to use, then change to listen to the same socket in the virtualhost configuration or in the override segment in .htaccess)

In the virtualhost configuration, that same listening socket must be used (the owner of the httpd process must have rights to talk to the proxy, so it could be set up as a per-user or per-group setting depending on how it was set up in the pool configuration).

Sample virtualhost, allows override of PHP version and access to /fpm-status

<VirtualHost *:80>
  <IfModule mpm_itk_module>
    AssignUserId #8001 #8001
  </IfModule>

  ServerName site8001.domain.com
  DocumentRoot /var/www/site8001/html

  <Directory /var/www/site8001/html>
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
  </Directory>
  <LocationMatch "/fpm-status">
    ProxyPass  "unix:/var/run/php/php8.2-fpm-8001.sock|fcgi://localhost/fpm-status"
  </LocationMatch>
  <FilesMatch ".php$">
    SetHandler "proxy:unix:/var/run/php/php8.2-fpm-8001.sock|fcgi://localhost/"
  </FilesMatch>

  ErrorLog /var/www/site8001/logs/error.log
  CustomLog /var/www/site8001/logs/access.log combined
</VirtualHost>

mpm_itk (used above) allows the Apache HTTPd process to be run by another user:

apt install libapache2-mpm-itk
a2enmod mpm_itk

The following were probably done when installing php-fpm

apt install libapache2-mod-fcgid
a2enmod proxy_fcgi

File permissions

When running both Apache httpd and PHP as a specific user, the files on the web site only need to have user read/write(when needed) access if they are owned by the user running the processes.
To make a permanent change to the umask used by PHP, add ‘UMask=0077’ to the ‘Service’ section of each PHP FPM service:

systemctl edit php8.2-fpm.service

Add:

[Service]
UMask=0077

Then reload systemctl daemon and restart the fpm service:

systemctl daemon-reload && systemctl restart php8.2-fpm

https://serverok.in/apache-virtualhost-separate-user-with-mpm-itk

How to download complete items from archive.org

Download all files attached to a item page at archive.org
Navigate to the item page you want to download all the files from.
Download the XML filelist (named as the item, get the file ending with “_files.xml”).
Parse the filelist for the files (quick and ugly):

grep "file name=" someitem_files.xml | sed s:\<file\ name=:\<a\ href=:g | sed s:\>:\>file\<\/a\>:g

This will keep the lines containing “file name=” and create a output only containing (relative, as in the file list) html links to each file.

Redirect the output to a file (I assume you know how), then download with wget:

wget -r -H -nc -np -nH --cut-dirs=1 -e robots=off -l1 -F -i someitem.items -B "https://archive.org/download/someitem/"

For more advanced downloading, I have created a set of script (not yet released) that allow downloads of a complete collection (of other item pages) or download of everything uploaded by a specific user. My scripts will also create ‘md5sum -c’ compatible lists from the _files.xml files, execute the checking and optionally delete corrupt files for re-downloading.

My floppy preservation methods using Kryoflux

This article documents my methods for preserving floppy disks. There is probably a better way that I haven’t thought about yet.
The downside of the methods described herein, is that the Kryoflux project is more or less abandoned, and the methods here are not as easy to implement with the widely available and supported Greaseweasle equivalent.

General guidelines

  • Always use a clean, known good floppy drive for preservation attempts.
  • For the first read attempt, use the GUI for simplicity and generating the logs without having to bother with the command line parameters.
  • If there’s a label on the disk, use it to identify the disk that is read so you later easily can find it to doing re-reads of failed tracks.
  • If the format is known, select that in the output format selection drop down.

Software used

I use the latest/last windows version of the dtc (Kryoflux) software that can be found on the Kryoflux download page. As (recently) the Linux version was also updated to the last one (3.00), it should be equally usable for my methods.
Within windows, I the use microsoft Ubuntu shell for all operations except the ‘dtc’ command which is run in a microsoft shell. If you find suitable alternatives to ‘grep’ (search in files), ‘split’ (split file into parts) and ‘cat’ (join file parts alphabetically into a single file), I see no reason that this couldn’t be done using only windows shell.

Guessing the disk format

Guessing the format and saving data as Kryoflux preservation raw files

All DS/DD DS/HD formats

dtc -ftest\track -i0 -i2 -i3 -i4 -i5 -i9 -i11 -i12 -d0 -p >test.log

Checking result (recognized formats)

grep ": OK" test.log

Examining the read results

Good reads:
Find disks which was read 100% ok on the first read of all tracks

grep -L bad *log|sort

Find disks without any non-recoverable read errors

grep -L failed *log|sort

Bad / incomplete reads:
Find disks which have at least one non-recoverable read error

grep -l failed *log|sort

Find disks which have at least one track below 80 indicated as unformatted

grep -l "^[^8]*<unformatted>" *log|sort

Re-reading bad tracks

If possible, use another, newly cleaned, disk drive to try to re-read the tracks that previously failed. The same method can also be used to combine two mastered disks with errors on different tracks if re-reading from the disk with the failed tracks still isn’t possible.

Use the track format verification options if you’re sure about the format (will do no damage, but adds extra info to the logs). -i2 (CT RAW) is selected as a verification format by the GUI, so I keep that and add Amiga (-i5) in the example.

Read one track at a time – this seem to increase the chance of correct results since the read head has to move directly to that location instead of just ”dragging” itself over the damaged floppy while it is rotating.

Find which tracks needs to be re-read

grep failed DiskID.log

Re-read tracks with errors save as Kryoflux preservation format and try to verify as possible format(s)
The parameters -s and -e sets the start and end track. Use the same value of both even if the tracks are located next to each other (see above).

If for example tracks 21, 66, 67, 68, 69 and 70 failed when reading the disk for the first time:

dtc -l15 -f"retry1\DiskID\track" -i0 -i2 -i5 -d0 -p -s21 -e21 -t50
dtc -l15 -f"retry1\DiskID\track" -i0 -i2 -i5 -d0 -p -s66 -e66 -t50
dtc -l15 -f"retry1\DiskID\track" -i0 -i2 -i5 -d0 -p -s67 -e67 -t50
dtc -l15 -f"retry1\DiskID\track" -i0 -i2 -i5 -d0 -p -s68 -e68 -t50
dtc -l15 -f"retry1\DiskID\track" -i0 -i2 -i5 -d0 -p -s69 -e69 -t50
dtc -l15 -f"retry1\DiskID\track" -i0 -i2 -i5 -d0 -p -s70 -e70 -t50

Take a note of which (if not all) of the tracks were recoverable using this read method. Even if a track fails to be read, a data file will be stored.

Methods of combining multiple reads into one

Using the raw files (Kryoflux preservation format) from both reads
If you have the raw files from the first read, copy them to another place and then copy the raw files from the new read into that folder (replace those from the first read).
Create the floppy disk image using this mix of source files from either the GUI or the command line.

Using the new raw files and an incomplete disk image from the first read
Some knowledge about the disk format is needed for this method. The most important parameter is the number of bytes per track (in the case of the Amiga it is 512*11*2, which is 11264).

Use the DTC GUI or command line to create the assumed floppy image type (Amiga in this case) from the raw data files. This image will be inomplete, and will contain only the re-read tracks.

Split the old and the new image into track-sized parts:
The files will (by default) be named xaa, xab etc, but -d changes this to x00, x01 … Also, the prefix (x) could be changed, but that depends on the implementation of the used split command. Safest is to split into two subdirectories and keep the original names like:

(current directory holds copies of both disk image files to combine)

mkdir old
cd old
split ../old.adf -b11264 -d
cd ..
mkdir new
cd new
split ../new.adf -b11264 -d

Copy the new (those that was correctly read) tracks into the ”old” folder:

cp x21 x66 x67 x68 x69 x70 ../old

Join (now mixed) content in ”old” as a new disk image file:

cd ../old
cat x* >>../combined.adf

That’s it. This guide has not been tested recently, but was just jotted down while preserving some badly damaged floppies about a year ago. My post in the Kryoflux forum:
Method for reading problematic disks (?)

Buffalo LS220D – lost drive (hiccup)

Yesterday I noticed that the LEDs were blinking amber on one of my LS220D boxes. My initial thought was that a disk had failed (it’s just a backup of my backup). Checked with the “NAS Navigator” application, and it stated that it was unable to mount the data array (md10) (I have not logged the full error message here, as I continued the attempts to solve the situation).

dmesg output

I logged in as root (see other posts) to check what had gone wrong.
‘dmesg’ revealed that a disk had been lost during smartctl (multiple repeats of the below content):

program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
Unable to handle kernel NULL pointer dereference at virtual address 000000a4
pgd = c27d4000
[000000a4] *pgd=0fe93831, *pte=00000000, *ppte=00000000
Internal error: Oops: 817 [#50]
Modules linked in: usblp usb_storage ohci_hcd ehci_hcd xhci_hcd usbcore usb_common
CPU: 0    Tainted: G      D       (3.3.4 #1)
PC is at sg_scsi_ioctl+0xe0/0x374
LR is at sg_scsi_ioctl+0xcc/0x374
pc : []    lr : []    psr: 60000013
sp : cafb5d58  ip : 00000000  fp : 00000024
r10: 00000006  r9 : c41d1860  r8 : 00000012
r7 : 00000000  r6 : 00000024  r5 : beee5550  r4 : beee5548
r3 : cafb4000  r2 : cafb5d58  r1 : 00000000  r0 : 00000000
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: 10c5387d  Table: 027d4019  DAC: 00000015
Process smartctl (pid: 1027, stack limit = 0xcafb42e8)
Stack: (0xcafb5d58 to 0xcafb6000)
5d40:          c057c2b8 60000013
5d60: c21f27f0 beee5548 c2274800 0000005d cafb5de4 00000000 c998edcc 00000004
5d80: c99800c8 c00a6e64 c9d034e0 00000028 c998edc8 00000029 c27d4000 c00a8fc0
5da0: 00000000 00000000 00000000 c998ed08 c2274800 56e6994b beee5a48 beee5548
5dc0: 0000005d 0000005d c2274800 c21f27f0 cafb4000 56e6994b beee7e34 beee5548
5de0: 0000005d 0000005d c2274800 c21f27f0 cafb4000 ffffffed beee7e34 c0245494
5e00: 00000053 fffffffd 00002006 00000024 beee5af8 beee5ae0 beee5ab8 00004e20
5e20: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
5e40: c27d4000 00000000 c27d4000 cb0023e0 c87f3d30 00000028 beee7e34 c00be67c
5e60: c27d4000 00000028 cafb5fb0 56e6994b 00000001 0000005d c8014040 beee5548
5e80: 0000005d c0245530 beee5548 0000005d 00000001 00000001 beee5548 c222c000
5ea0: c8014040 c02a6284 beee5548 beee5548 c8014040 c2274800 00000001 0000005d
5ec0: 00000000 c02422a0 beee5548 c0242be0 00000000 cafb5f78 00000001 c2949000
5ee0: ffffff9c c8014040 00000000 00000007 c054ff34 00039db8 cafb5fb0 beee5548
5f00: c21e0470 00000003 00000003 c000e3c8 cafb4000 00000000 beee7e34 c00e0060
5f20: 00000000 00000000 cf34be00 2c1b812a 5e6a6136 2c1b812a cf1a2548 00000000
5f40: 00000000 00000000 00000003 00000003 c95a2ec0 c2949000 c95a2ec8 00000020
5f60: 00000003 c95a2ec0 beee5548 00000001 00000003 c000e3c8 cafb4000 00000000
5f80: beee7e34 c00e010c 00000003 00000000 beee5548 beee5548 0006d614 beee5a8c
5fa0: 00000036 c000e200 beee5548 0006d614 00000003 00000001 beee5548 00000000
5fc0: beee5548 0006d614 beee5a8c 00000036 00000000 00000003 00000006 beee7e34
5fe0: beee5ae0 beee5540 00039688 b6da5cec 80000010 00000003 cfcfcfcf 00000014
[] (sg_scsi_ioctl+0xe0/0x374) from [] (scsi_cmd_ioctl+0x39c/0x3fc)
[] (scsi_cmd_ioctl+0x39c/0x3fc) from [] (scsi_cmd_blk_ioctl+0x3c/0x44)
[] (scsi_cmd_blk_ioctl+0x3c/0x44) from [] (sd_ioctl+0x8c/0xb8)
[] (sd_ioctl+0x8c/0xb8) from [] (__blkdev_driver_ioctl+0x20/0x28)
[] (__blkdev_driver_ioctl+0x20/0x28) from [] (blkdev_ioctl+0x670/0x6c0)
[] (blkdev_ioctl+0x670/0x6c0) from [] (do_vfs_ioctl+0x49c/0x514)
[] (do_vfs_ioctl+0x49c/0x514) from [] (sys_ioctl+0x34/0x58)
[] (sys_ioctl+0x34/0x58) from [] (ret_fast_syscall+0x0/0x30)
Code: e1a0200d e7d3a2a8 e3c23d7f e3c3303f (e1c0aab4)
---[ end trace 660c9d3c9b4a9034 ]---

fdisk output

Using ‘fdisk’ (incorrect for this NAS), I listed the partitions on /dev/sda and /dev/sdb (nothing about /dev/sda):

[root@BUFFALO-4 ~]# fdisk -l /dev/sda
[root@BUFFALO-4 ~]# fdisk -l /dev/sdb

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdb: 4000.8 GB, 4000787030016 bytes
255 heads, 63 sectors/track, 486401 cylinders, total 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1  4294967295  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.

smartctl output

[root@BUFFALO-4 ~]# smartctl --scan
/dev/sda -d scsi # /dev/sda, SCSI device
/dev/sdb -d scsi # /dev/sdb, SCSI device

[root@BUFFALO-4 ~]# smartctl --all /dev/sda
smartctl 6.3 2014-07-26 r3976 [armv7l-linux-3.3.4] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

Segmentation fault

[root@BUFFALO-4 ~]# smartctl --all /dev/sdb
smartctl 6.3 2014-07-26 r3976 [armv7l-linux-3.3.4] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green (AF, SATA 6Gb/s)
Device Model:     WDC WD40EZRX-22SPEB0
Serial Number:    WD-WCC4E1UUZH74
LU WWN Device Id: 5 0014ee 2b768eeb4
Firmware Version: 80.00A80
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Thu Jul 14 12:10:33 2022 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
          was completed without error.
          Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
          without error or no self-test has ever
          been run.
Total time to complete Offline
data collection: (52320) seconds.
Offline data collection
capabilities:     (0x7b) SMART execute Offline immediate.
          Auto Offline data collection on/off support.
          Suspend Offline collection upon new
          command.
          Offline surface scan supported.
          Self-test supported.
          Conveyance Self-test supported.
          Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
          power-saving mode.
          Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
          General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 523) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x7035) SCT Status supported.
          SCT Feature Control supported.
          SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   196   187   021    Pre-fail  Always       -       7183
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       36
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   055   054   000    Old_age   Always       -       33525
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       36
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       28
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       7866202
194 Temperature_Celsius     0x0022   113   103   000    Old_age   Always       -       39
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       1

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status   Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%         8         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Nothing more to do than to reboot.

After reboot

The storage array was still not mounted, smartctl could now find /dev/sda:

[root@BUFFALO-4 ~]# df -h
Filesystem Size      Used Available Use% Mounted on
udev      10.0M         0     10.0M   0% /dev
/dev/md1   4.7G    766.8M      3.7G  17% /
tmpfs    121.1M     84.0K    121.0M   0% /tmp
/dev/ram1 15.0M    100.0K     14.9M   1% /mnt/ram
/dev/md0 968.7M    216.4M    752.2M  22% /boot

[root@BUFFALO-4 ~]# smartctl --all /dev/sda
smartctl 6.3 2014-07-26 r3976 [armv7l-linux-3.3.4] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green (AF, SATA 6Gb/s)
Device Model:     WDC WD40EZRX-22SPEB0
Serial Number:    WD-WCC4E1XUDU4T
LU WWN Device Id: 5 0014ee 20cbde2d7
Firmware Version: 80.00A80
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Thu Jul 14 12:13:56 2022 CEST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x84) Offline data collection activity
          was suspended by an interrupting command from host.
          Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
          without error or no self-test has ever
          been run.
Total time to complete Offline
data collection: (52560) seconds.
Offline data collection
capabilities:     (0x7b) SMART execute Offline immediate.
          Auto Offline data collection on/off support.
          Suspend Offline collection upon new
          command.
          Offline surface scan supported.
          Self-test supported.
          Conveyance Self-test supported.
          Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
          power-saving mode.
          Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
          General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 526) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x7035) SCT Status supported.
          SCT Feature Control supported.
          SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   250   204   021    Pre-fail  Always       -       4500
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       38
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   053   051   000    Old_age   Always       -       34713
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       38
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       30
193 Load_Cycle_Count        0x0032   001   001   000    Old_age   Always       -       7823449
194 Temperature_Celsius     0x0022   122   106   000    Old_age   Always       -       30
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       13
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       11
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       14

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status   Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%         8         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Partition table after reboot

Now when both disks are in place again, I ran the (correct) command to list the partitions on all drives:

[root@BUFFALO-4 ~]# parted -l /dev/sdb
Model: ATA WDC WD40EZRX-22S (scsi)
Disk /dev/sda: 4001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
 1      17.4kB  1024MB  1024MB  ext3         primary
 2      1024MB  6144MB  5119MBprimary
 3      6144MB  6144MB  394kB primary  bios_grub
 4      6144MB  6144MB  512B  primary
 5      6144MB  7168MB  1024MBprimary
 6      7168MB  3992GB  3985GBprimary


Model: ATA WDC WD40EZRX-22S (scsi)
Disk /dev/sdb: 4001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
 1      17.4kB  1024MB  1024MB  ext3         primary
 2      1024MB  6144MB  5119MBprimary
 3      6144MB  6144MB  394kB primary  bios_grub
 4      6144MB  6144MB  512B  primary
 5      6144MB  7168MB  1024MBprimary
 6      7168MB  3992GB  3985GBprimary

...

Looks ok, so I tried mounting /dev/md10:

root@BUFFALO-4 ~]# mount /dev/md10 /mnt/array1/
[root@BUFFALO-4 ~]# df -h
Filesystem Size      Used Available Use% Mounted on
udev      10.0M         0     10.0M   0% /dev
/dev/md1   4.7G    766.8M      3.7G  17% /
tmpfs    121.1M     84.0K    121.0M   0% /tmp
/dev/ram1 15.0M    100.0K     14.9M   1% /mnt/ram
/dev/md0 968.7M    216.4M    752.2M  22% /boot
/dev/md10  7.2T      5.7T      1.6T  79% /mnt/array1
[root@BUFFALO-4 ~]# ls /mnt/array1/
backup/         buffalo_fix.sh* share/          spool/
[root@BUFFALO-4 ~]# ls /mnt/array1/share/
acp_commander/    buff4_public.txt  buff4_share.txt   buff4_web.txt

Checking the file system for errors

As I was able to mount the partition, I did a file system check after unmounting it:

[root@BUFFALO-4 ~]# xfs_repair /dev/md10
Phase 1 - find and verify superblock...
Not enough RAM available for repair to enable prefetching.
This will be _slow_.
You need at least 1227MB RAM to run with prefetching enabled.
Phase 2 - using internal log
        - zero log...
        - scan filesystem freespace and inode maps...
        - found root inode chunk
Phase 3 - for each AG...
        - scan and clear agi unlinked lists...
        - process known inodes and perform inode discovery...
        - agno = 0
        - agno = 1
...
        - agno = 30
        - agno = 31
        - process newly discovered inodes...
Phase 4 - check for duplicate blocks...
        - setting up duplicate extent list...
        - check for inodes claiming duplicate blocks...
        - agno = 0
        - agno = 1
...
        - agno = 30
        - agno = 31
Phase 5 - rebuild AG headers and trees...
        - reset superblock...
Phase 6 - check inode connectivity...
        - resetting contents of realtime bitmap and summary inodes
        - traversing filesystem ...
doubling cache size to 1024
        - traversal finished ...
        - moving disconnected inodes to lost+found ...
Phase 7 - verify and correct link counts...
done
[root@BUFFALO-4 ~]# mount /dev/md10 /mnt/array1
[root@BUFFALO-4 ~]# ls /mnt/array1/
backup/         buffalo_fix.sh* share/          spool/

Another reboot, then checking to find out that md10 was still not mounted.
The error in NAS Navigator is: “E14:RAID array 1 could not be mounted. (2022/07/14 12:36:18)”

Time to check ‘dmesg’ again:

md/raid1:md2: active with 1 out of 2 mirrors
md2: detected capacity change from 0 to 1023410176
md: md1 stopped.
md: bind
md/raid1:md1: active with 1 out of 2 mirrors
md1: detected capacity change from 0 to 5114888192
md: md0 stopped.
md: bind
md/raid1:md0: active with 1 out of 2 mirrors
md0: detected capacity change from 0 to 1023868928
 md0: unknown partition table
kjournald starting.  Commit interval 5 seconds
EXT3-fs (md0): using internal journal
EXT3-fs (md0): mounted filesystem with writeback data mode
 md1: unknown partition table
kjournald starting.  Commit interval 5 seconds
EXT3-fs (md1): using internal journal
EXT3-fs (md1): mounted filesystem with writeback data mode
kjournald starting.  Commit interval 5 seconds
EXT3-fs (md1): using internal journal
EXT3-fs (md1): mounted filesystem with writeback data mode
 md2: unknown partition table
Adding 999420k swap on /dev/md2.  Priority:-1 extents:1 across:999420k
kjournald starting.  Commit interval 5 seconds
EXT3-fs (md0): using internal journal
EXT3-fs (md0): mounted filesystem with writeback data mode

The above shows that md0, md1 and md2 went up, but are missing its mirror partition (this from /dev/sda that disappeared).
Further down in dmesg output

md: md10 stopped.
md: bind
md: bind
md/raid0:md10: md_size is 15565748224 sectors.
md: RAID0 configuration for md10 - 1 zone
md: zone0=[sda6/sdb6]
      zone-offset=         0KB, device-offset=         0KB, size=7782874112KB

md10: detected capacity change from 0 to 7969663090688
 md10: unknown partition table
XFS (md10): Mounting Filesystem
XFS (md10): Ending clean mount
XFS (md10): Quotacheck needed: Please wait.
XFS (md10): Quotacheck: Done.
udevd[3963]: starting version 174
md: cannot remove active disk sda6 from md10 ...
[root@BUFFALO-4 ~]# mount /dev/md10 /mnt/array1/
[root@BUFFALO-4 ~]# ls -l /mnt/array1/
total 4
drwxrwxrwx    3 root     root            21 Dec 14  2019 backup/
-rwx------    1 root     root           571 Oct 14  2018 buffalo_fix.sh*
drwxrwxrwx    3 root     root            91 Sep 16  2019 share/
drwxr-xr-x    2 root     root             6 Oct 21  2016 spool/

What the h… “cannot remove active disk sda6 from md10”

Checking md raid status

[root@BUFFALO-4 ~]# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md10 : active raid0 sda6[0] sdb6[1]
      7782874112 blocks super 1.2 512k chunks

md0 : active raid1 sdb1[1]
      999872 blocks [2/1] [_U]

md1 : active raid1 sdb2[1]
      4995008 blocks super 1.2 [2/1] [_U]

md2 : active raid1 sdb5[1]
      999424 blocks super 1.2 [2/1] [_U]

unused devices: 
[root@BUFFALO-4 ~]# mdadm --detail /dev/md10
/dev/md10:
        Version : 1.2
  Creation Time : Fri Oct 21 15:58:46 2016
     Raid Level : raid0
     Array Size : 7782874112 (7422.33 GiB 7969.66 GB)
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent

    Update Time : Fri Oct 21 15:58:46 2016
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

     Chunk Size : 512K

           Name : LS220D896:10
           UUID : 5ed0c596:60b32df6:9ac4cd3a:59c3ddbc
         Events : 0

    Number   Major   Minor   RaidDevice State
       0       8        6        0      active sync   /dev/sda6
       1       8       22        1      active sync   /dev/sdb6

So here, md10 is fully working and md0, md1 and md2 are missing their second device. Simple to correct, just adding them back:

[root@BUFFALO-4 ~]# mdadm --manage /dev/md0 --add /dev/sda1
mdadm: added /dev/sda1
[root@BUFFALO-4 ~]# mdadm --manage /dev/md1 --add /dev/sda2
mdadm: added /dev/sda2
[root@BUFFALO-4 ~]# mdadm --manage /dev/md2 --add /dev/sda5
mdadm: added /dev/sda5
[root@BUFFALO-4 ~]# cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md10 : active raid0 sda6[0] sdb6[1]
      7782874112 blocks super 1.2 512k chunks

md0 : active raid1 sda1[0] sdb1[1]
      999872 blocks [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md1 : active raid1 sda2[2] sdb2[1]
      4995008 blocks super 1.2 [2/1] [_U]
      [====>................]  recovery = 24.2% (1212672/4995008) finish=1.2min speed=48506K/sec

md2 : active raid1 sda5[2] sdb5[1]
      999424 blocks super 1.2 [2/1] [_U]
        resync=DELAYED

unused devices: 

Some time later, sync was finished, and I rebooted again. Finally, after this reboot /dev/md10 is automatically mounted to /mnt/array1 again.

Problem solved 🙂

smartctl notes

The values of attributes 5, 197 and 198 should be zero for a healthy drive, so one disk in the NAS is actually failing, but the cause of the hiccup (disconnect) was a core dump by smatctl weekly scan.

  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       13
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      11

Beating CGNAT with reverse SSH tunnels

Because of my current internet provider which refuse to give me a public IP (providing me with only CGNAT), I can no longer access my stuff at home as I wish. This post describes my workaround for the problem.

External access server

Used to connect the tunnel from the inside to, and to connect to from the outside. I use a Oracle Free tier WM for this.

On the access server, I have set up a (normal) user account for the tunnel, then created a private key using Puttygen and added the public key to the .ssh/authorized_keys file for the user.
/etc/ssh/sshd_config needs to be modified by adding the line
GatewayPorts clientspecified
Do not forget to check that you are able to login to the server using the user set up for this purpose (set private key login in Putty, or use ssh -i with that key)

Firewall on the access server
If the access server is behind other firewalls, you need to open the port(s) you want to connect to.
For the Oracle VMs, this is done via the web UI:
Virtual Cloud Networks, click the VCN name, click the subnet name, click the security list (“Default security list” unless you have done it the recommended way to create separate security lists). Then (at last), “Add Ingress Rules”:
Source CIDR: 0.0.0.0/0 (unless you want to limit, but just for testing this will allow everyone to connect)
Source port: blank
Destination port: the port you want to connect to. An unprivileged user (the SSH user account) can only use ports 1025 and up.

The same port(s) also need to be opened on the access server itself.
I prefer using firewalld for this:
# firewall-cmd –zone=public –permanent –add-port={your-port-number-here}/tcp
# firewall-cmd –reload

GUI for plink (from Putty) to keep a reverse SSH tunnel open

https://myentunnel.informer.com/download/
Download the file (myentunnel_setup-3.6.1.exe), install it and then replace the included plink.exe with the current version included by the Putty installation.
Putty location: C:\Program Files\PuTTY
myEntunnel location: C:\Program Files\MyEnTunnel
Files used for the default profile: localports.txt (blank), remoteports.txt, keyfile.ppk (used for connecting to the server)
Ref: https://superuser.com/questions/235395/automatic-ssh-tunneling-from-windows

MyEntunnel configuration

Settings tab:
The obvious section, server (name or IP) and username, I suppose you know what to fill in there 🙂
No passphrase is needed, since we’re connecting with a private key only (this is also required by Oracle VMs)

As I forgot to note down the default settings, I just provide a snapshot of the settings I have in use (most are default values):

Tunnels tab:
Only the remote side needs to be filled in, syntax as the description below the input fields (per tunnel to create):
[listen-IP:]listen-port:host:port
Where listen-IP is the LOCAL ip of the access server (a private IP address if behind NAT as with the Oracle WMs, which are usually by default on the 10.x.x.x network)
listen-port is the port opened in the inside and outside firewalls (the port on which you will access the inside stuff on)
host is the inside host, can be localhost for the computer running MyEntunnel, or any other host reachable from that computer.
port is the port on the inside host
A complete line could look like this:
10.0.0.3:18180:192.168.101.180:80
(will access port 80 on the inside host IP 192.168.101.180, when going through the access server at port 18180)