David Heinemannhttps://dheinemann.com/assets/about.png2022-05-19T22:23:34+10:00David HeinemannBroken Audio in Fedora 35 with Wireplumber 0.4.102022-05-16T00:00:00+10:002022-05-16T00:00:00+10:00https://dheinemann.com/posts/2022-05-16-broken-audio-fedora-35-with-wireplumber-0-4-10<p> I must be unlucky because I've encountered my second breakage on Fedora 35 in just a couple of weeks! Other than this, Fedora has been the most stable and hassle-free Linux distros I've ever used, so I'm sure these are just outliers. In fact, this one is partially my own fault. Maybe you're in the same boat as me? </p> <p> A couple of days ago I upgraded Wireplumber to 0.4.10-1 on my Fedora 35 desktop. After restarting, Gnome no longer detected my audio devices (speakers/microphone) and I was unable to play or record audio. </p> <div style="text-align: center;"> <figure> <img src="https://dheinemann.com/assets/wireplumbernoaudio.png" alt="Screnshot of Gnome Settings showing that the list of Output Devices is empty" style="max-width: 100%"/> <figcaption> The list of available Output Devices in Gnome Settings is empty </figcaption> </figure> </div> <p> Despite this, systemctl shows that the relevant PipeWire services are all running. pipewire-pulse shows a 'pactl' error, but I don't think it's related. </p> <figure> <figcaption class="code">Status of my Pipewire services</figcaption> <pre class="code">[david@Abraxas ~]$ systemctl --user status pipewire* ● pipewire.socket - PipeWire Multimedia System Socket Loaded: loaded (/usr/lib/systemd/user/pipewire.socket; enabled; vendor preset: enabled) Active: active (running) since Mon 2022-05-16 20:43:44 AEST; 2min 51s ago Triggers: ● pipewire.service Listen: /run/user/1000/pipewire-0 (Stream) CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/pipewire.socket May 16 20:43:44 Abraxas systemd[2251]: Listening on PipeWire Multimedia System Socket. ● pipewire-pulse.service - PipeWire PulseAudio Loaded: loaded (/usr/lib/systemd/user/pipewire-pulse.service; disabled; vendor preset: disabled) Active: active (running) since Mon 2022-05-16 20:43:44 AEST; 2min 51s ago TriggeredBy: ● pipewire-pulse.socket Main PID: 12427 (pipewire-pulse) Tasks: 2 (limit: 38220) Memory: 1.5M CPU: 9ms CGroup: /user.slice/user-1000.slice/user@1000.service/session.slice/pipewire-pulse.service └─12427 /usr/bin/pipewire-pulse May 16 20:43:44 Abraxas systemd[2251]: Started PipeWire PulseAudio. May 16 20:43:44 Abraxas pipewire-pulse[12442]: pw.conf: execvp error 'pactl': No such file or directory ● pipewire.service - PipeWire Multimedia Service Loaded: loaded (/usr/lib/systemd/user/pipewire.service; disabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/user/pipewire.service.d └─00-uresourced.conf Active: active (running) since Mon 2022-05-16 20:43:44 AEST; 2min 51s ago TriggeredBy: ● pipewire.socket Main PID: 12425 (pipewire) Tasks: 2 (limit: 38220) Memory: 1.7M CPU: 12ms CGroup: /user.slice/user-1000.slice/user@1000.service/session.slice/pipewire.service └─12425 /usr/bin/pipewire May 16 20:43:44 Abraxas systemd[2251]: Started PipeWire Multimedia Service. ● pipewire-pulse.socket - PipeWire PulseAudio Loaded: loaded (/usr/lib/systemd/user/pipewire-pulse.socket; enabled; vendor preset: enabled) Active: active (running) since Mon 2022-05-16 20:43:44 AEST; 2min 51s ago Triggers: ● pipewire-pulse.service Listen: /run/user/1000/pulse/native (Stream) CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/pipewire-pulse.socket May 16 20:43:44 Abraxas systemd[2251]: Listening on PipeWire PulseAudio. </pre> </figure> <h2>Workaround</h2> <p> I tried downgrading Wireplumber to 0.4.4-2, then restarting the PipeWire services: </p> <figure> <pre class="code">sudo dnf downgrade wireplumber systemctl --user restart pipewire*</pre> </figure> <p> Sure enough, this fixed the issue and audio worked again. But that's not a <i>permanent</i> solution. Why did this occur in the first place? </p> <h2>The Problem</h2> <p> The issue seems to be that Wireplumber 0.4.10 includes important config changes that are required for audio to work. However, if you have manually modified your Wireplumber configs at any point, <b>you won't receive these config changes automatically</b>! I assume this is a distro feature to prevent users' customisations from being overwritten by package upgrades. In my case, I had modified /etc/wireplumber/main.lua.d/50-alsa-config.lua to fix <a href="https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/1776#note_1146196">an audio pop issue</a><sup>&dagger;</sup>. </p> <h2>The Solution</h2> <p> To restore audio, you must update your Wireplumber configs, and then restart your PipeWire services to apply them. There are two options. </p> <h3>Option 1: Manually Copy the Required Changes</h3> <p> To preserve your Wireplumber customisations, you can manually copy the necessary changes into your existing Wireplumber configs in /etc/wireplumber/main.lua.d/ . A full list of required changes in each file can be found in <a href="https://gitlab.freedesktop.org/pipewire/wireplumber/-/commit/3bdacc414e540cdbe003cc6276b89ec2397f0c66">this Wireplumber commit</a>. </p> In my case, I had to add this line to the start of /etc/wireplumber/main.lua.d/50-alsa-config.lua: </p> <figure> <pre class="code">alsa_monitor.enabled = true</pre> </figure> <p> After making your changes, restart the PipeWire services: </p> <figure> <pre class="code">systemctl --user restart pipewire*</pre> </figure> <h3>Option 2: Restore Wireplumber Defaults</h3> <p> If you're unsure what changes to apply or where, you can simply restore your Wireplumber configs to their defaults. Copy the default configs from /usr/share/wireplumber/main.lua.d/ to /etc/wireplumber/main.lua.d/ . Don't forget to take a backup first! </p> <p> After making your changes, restart the PipeWire services: </p> <figure> <pre class="code">systemctl --user restart pipewire*</pre> </figure> <p> Thanks to grumpey0 for <a href="https://bugzilla.redhat.com/show_bug.cgi?id=2086190#c0">highlighting the issue and resolution</a> on the Fedora bug tracker! You made this really easy for me to troubleshoot and fix. </p> <h2>Further Information</h2> <ul> <li><a href="https://gitlab.freedesktop.org/pipewire/wireplumber/-/issues/254">Wireplumber bug report #254</a></li> <li><a href="https://bugzilla.redhat.com/show_bug.cgi?id=2086190">Fedora 35 bug report #2086190</a></li> </ul> <hr> <div class="footnote"> <p> <sup>&dagger;</sup> I should probably find a way to do this in my home directory's configs instead, but that's a job for another day. </p> </div>Broken SMB Shares in Fedora 352022-05-16T00:00:00+10:002022-05-02T00:00:00+10:00https://dheinemann.com/posts/2022-05-02-working-around-broken-smb-shares-fedora-35<p> This issue has now been resolved via package upgrades. If you are affected, upgrading your packages should fix the issue. Original post below&hellip; </p> <hr> <p> SMB shares are currently broken in Gnome on Fedora 35, and potentially other distributions. If you have upgraded your Fedora 35 packages since late April 2022, you may encounter this error message when attempting to open an SMB share in Nautilus: </p> <div style="text-align: center;"> <figure> <img src="https://dheinemann.com/assets/smberror.png" alt="Unable to access location Failed to mount Windows share: Invalid argument" style="max-width: 100%"/> <figcaption> Unable to access location<br>Failed to mount Windows share: Invalid argument </figcaption> </figure> </div> <p> However, manual testing with smbclient shows that the SMB connection is actually fine, and I can connect to my SMB share without any issues: </p> <figure> <figcaption class="code">Testing a Samba connection with smbclient</figcaption> <pre class="code">[david@Abraxas ~]$ smbclient \\\\filepile\\archives -U david Password for [SAMBA\david]: Try "help" to get a list of possible commands. smb: \> dir . D 0 Mon Aug 23 19:52:25 2021 .. D 0 Sat Sep 18 13:04:36 2021 Personal D 0 Mon Aug 23 19:51:49 2021 General D 0 Fri Mar 11 20:24:49 2022 7810221624 blocks of size 1024. 4482518988 blocks available </pre> </figure> <h2>The Problem</h2> <p> The root problem affecting Nautilus is a compatibility issue between gvfs 1.50.0 (and earlier) and these Samba versions: </p> <ul> <li>Samba 4.14.13 (and later)</li> <li>Samba 4.15.7 (and later)</li> <li>Samba 4.16.0 (and later)</li> </ul> <p> Fedora 35 uses gvfs 1.48.1, and was affected when Samba 4.15.7 was rolled out late April. Fedora 36 beta was using gvfs 1.50.0, and was similarly affected by Samba 4.16. However, this has already been fixed by upgrading gvfs to 1.50.1. </p> <h2>The Solution</h2> <p> gvfs 1.50.1 includes Gnome's <a href="https://gitlab.gnome.org/GNOME/gvfs/-/merge_requests/138">official fix</a> for this issue. If gvfs 1.50.1 is available to you (e.g. Fedora 36 beta users), consider upgrading to it. </p> <p> To my knowledge, gvfs 1.50.1 won't be made available for Fedora 35. However, the fix is being backported by the Fedora maintainers as <a href="https://koji.fedoraproject.org/koji/buildinfo?buildID=1963021">gvfs-1.48.1-3</a>. It will be available in the stable package repository shortly. </p> <h2>Workaround</h2> <p> If you can't upgrade gvfs yet, consider downgrading your Samba version as a workaround. In Fedora 35, you can downgrade your Samba packages to 4.15.0 using dnf: </p> <figure> <pre class="code">sudo dnf downgrade samba</pre> </figure> <p> After downgrading to a compatible version, you should be able to open SMB shares in Nautilus again. </p> <p> To avoid accidentally upgrading Samba until the gvfs fix is available, most package managers include an exclusion option. For example, Fedora 35 users can install all package updates <i>except</i> for Samba using the <code>--exclude</code> argument with dnf: </p> <figure> <pre class="code">sudo dnf upgrade --exclude=*samba*,*smb*</pre> </figure> <h2>Further Information</h2> <ul> <li><a href="https://bugzilla.samba.org/show_bug.cgi?id=14983">Samba bug report #14983</a></li> <li><a href="https://gitlab.gnome.org/GNOME/gvfs/-/issues/611">gvfs bug report #611</a></li> <li><a href="https://bugzilla.redhat.com/show_bug.cgi?id=2068976">Fedora 36 bug report #2068976</a></li> <li><a href="https://bodhi.fedoraproject.org/updates/FEDORA-2022-45c9d57591">Status of gvfs-1.48.1-3 package for Fedora 35</a></li> </ul>A Survey of Native Programming Languages2022-04-15T00:00:00+10:002022-04-15T00:00:00+10:00https://dheinemann.com/posts/2022-04-15-survey-of-native-languages<p> In 2021, I started learning C, a goal I have had for a long time. It brought a level of excitement back to programming that I haven't felt since I was a complete beginner. I built <a href="2021-06-26-announcing-replicalc">Replicalc</a>, a REPL-based calculator, experimented with developing <a href="https://github.com/dHeinemann/Apocrypha-Engine">a text adventure engine</a>, and had a lot of fun. I always planned to do more with these projects, but eventually ran out of steam because&hellip;C is a pain. </p> <p> I chose to develop these projects in C because I wanted to run them on both Linux and MS-DOS. That leaves few options, and C is likely the best documented and supported out of all of them. However, the reality is that C really sucks for application development, even on a small scale. It's a much slower, methodical, and error-prone process than I'd like it to be. I've enjoyed it and learned a lot from it, but it isn't practical enough for my purposes. It's time to look for something else. </p> <p> What I really want is a language where I can be immediately productive and not have to worry too much about details like memory allocation. C, C++, and likely D are all too low-level for what I'm looking for. However, I don't want my software to require installation of a runtime or interpreter either, so that rules out popular high-level languages such as C#, Python, etc. Which languages lie in between these two groups? </p> <p> Armed with a list of compiled languages <a href="https://en.wikipedia.org/wiki/Compiled_language#Languages">from Wikipedia</a>, I began looking through the options to see what suits my requirements. This blog post is an overview of the languages I've looked at, with my thoughts and impressions&mdash;it isn't intended to be an objective review. </p> <div class="contents" style="width: 180px;"> <b>Contents</b> <ul> <li><a href="#rust">Rust</a></li> <li><a href="#zig">Zig</a></li> <li><a href="#purebasic">PureBasic</a></li> <li><a href="#freebasic">FreeBASIC</a></li> <li><a href="#pascal">Pascal</a></li> <li><a href="#nim">Nim</a></li> <li><a href="#go">Go</a></li> <li><a href="#conclusion">Conclusion</a></li> </ul> </div> <h2 id="rust">Rust</h2> <p> <a href="https://www.rust-lang.org/">Rust</a> is a systems programming language that guarantees memory safety at compile-time&mdash;no garbage collector required. This makes it great for applications where security and reliability are essential, and it's lightning-fast. It's no wonder it's been voted the #1 most-loved programming language six years in a row on the annual Stack Overflow survey! </p> <p> The first thing I noticed when I started diving into Rust is just how good the available documentation is. The Rust Book is thorough and well-written. Beyond that, there's an endless number of community guides and published books. Rust has experienced a surge in popularity over the past 5-odd years, so there's a massive community writing documentation and contributing to the Rust ecosystem. </p> <p> On top of that, the development tools are outstanding, at least where VSCode is concerned. Its IntelliSense contains not only function and argument names, but also detailed descriptions and example code. There's a lot to like. </p> <p> However, Rust has a steep learning curve. It's not insurmountable, but it's a different way of thinking to what I'm used to, and it's a bit off-putting. When it comes to my personal projects, I want a language that's dead-simple so that there are no barriers to me sitting down and writing code. Rust has a lot of potential, but it's decidedly <i>not</i> simple. I'd love to use it more in the future, but it's not the best choice for my current personal projects. </p> <h3>What I Like</h3> <ul> <li>Guaranteed memory safety</li> <li>Large community</li> <li>Excellent documentation</li> </ul> <h3>What I Don't Like</h3> <ul> <li>Complex, has a much steeper learning curve than most languages</li> <li>Requires patience to learn</li> </ul> <h3>Links</h3> <ul> <li><a href="https://www.rust-lang.org/">Rust Website</a></li> <li><a href="https://doc.rust-lang.org/book/">The Rust Book</a></li> </ul> <h2 id="zig">Zig</h2> <p> <a href="https://ziglang.org/">Zig</a> shows a lot of promise. It aims to provide a high level of safety while also remaining as simple as possible. In this way, it fills the middle-ground between C and Rust. It also has first-class support for integrating with existing C libraries, which should make it easy to use libraries like GTK. </p> <p> I thought Zig would be a good choice for me. Safer than C, and simpler than Rust. However, I really don't like its syntax. It uses some odd choices that I can't get used to. For example: <ul> <li> <b>Inconsistency:</b> All documentation comments start with <code>///</code>; unless they are top-level documentation comments, in which case they must start with <code>//!</code>. </li> <li><b>Weird conventions:</b> In a multi-line string, each line must be prefixed with <code>\\</code>.</li> <li> <b>Seemingly missing features:</b> There is no conventional <code>for</code> loop for iterating over a number range. Instead, the best practice <a href="https://github.com/ziglang/zig/issues/139">seems to be</a> to wrap a <code>while</code> loop in its own block like this: <figure> <pre class="code">{var i: i32 = 0; while (i < 10; i += 1) { %%stderr.printf("%i\n", i); }}</pre> </figure> Why would that be considered a better choice than this? <figure> <pre class="code">for (var i: i32 = 0; i < 10; i += 1) { %%stderr.printf("%i\n", i); }</pre> </figure> There's probably a good answer&mdash;Zig seems to be well thought-out&mdash;but I don't know what it is. </li> </ul> </p> <p> I also feel like the documentation is a bit lacking. In its current form, it's more like an information dump than a well-organised manual. This is understandable because Zig is still very young and yet to reach version 1.0. I'm sure we'll see this improve as it matures. But in the meantime, I found it unwieldy and difficult to find what I'm looking for. </p> <p> In summary, I like where Zig is going. However, the syntax feels somewhat alien to me. Between that and the current documentation, I don't have the patience to thoroughly dive into it now. Maybe I'll revisit it in the future. </p> <h3>What I Like</h3> <ul> <li>Achieves a strong balance between safety and simplicity, without using a garbage collector</li> </ul> <h3>What I Don't Like</h3> <ul> <li>Awkward syntax</li> <li>No conventional <code>for</code> loop</li> <li>Documentation isn't as organised as I'd like</li> </ul> <h3>Links</h3> <ul> <li><a href="https://ziglang.org/">Zig Website</a></li> <li><a href="https://ziglang.org/learn/overview/">Zig In-Depth Overview</a></li> <li><a href="https://ziglang.org/learn/why_zig_rust_d_cpp/">Why Zig When There is Already C++, D, and Rust?</a></li> <li><a href="https://ziglang.org/documentation/master/#Case-Study-print-in-Zig">Language Reference</a></li> <li><a href="https://ziglang.org/documentation/master/std/">Standard Library Documentation</a></li> <li><a href="https://ziglearn.org/">Zig Learn</a></li> <li><a href="https://www.forrestthewoods.com/blog/failing-to-learn-zig-via-advent-of-code/">Failing to Learn Zig via Advent of Code - Forrest Smith</a></li> </ul> <h2 id="purebasic">PureBasic</h2> <p> <a href="https://www.purebasic.com">PureBasic</a> ticks a lot of the boxes I'm looking for. It compiles to native, it's high-level, and it's really simple. </p> <p> What I like most about PureBasic is how intuitive it is. I was able to get productive almost immediately, and spent the afternoon developing a console-based Wordle clone. <a href="https://www.purebasic.com/documentation/">The documentation</a> is well laid out, although more examples would be welcome. The standard library has plenty of features, including an easy-to-use (if limited) <a href="https://www.purebasic.com/documentation/window/index.html">GUI library</a>. PureBasic's included IDE is decent, and features syntax highlighting, GUI debugging, and a form designer&mdash;everything you need to get started. </p> <p> However, PureBasic is not without its flaws. The most glaring is that it's a proprietary and commercial product. I wouldn't have even considered looking at PureBasic if I didn't already have a license. I bought one in 2011 and completely forgot about it, so I'm approaching PureBasic now for the first time. </p> </p> The license is affordable and include updates for life. However, I couldn't justify publishing my code under a Free Software license if it required a commercial compiler to use. In fact, it's difficult to justify paying for a compiler at all in 2022, when they are nearly all free of charge and Free Software. </p> <p> There are also some curious absences. For example: <ul> <li> In console applications, there's no way to get the current cursor location. You can <i>set</i> the location&mdash;but you can't get it. </li> <li> In windowed applications, there's no way to bind a procedure to a keypress unless you either: <ul> <li> Call the OS native API, which means sacrificing cross-platform compatibility; or </li> <li> Create a menu bar, add a menu item, bind the menu item to a callback, and then bind each individual key as a shortcut to that menu item. </li> </ul> You can see examples of both approaches in <a href="https://www.purebasic.fr/english/viewtopic.php?p=418002">this forum thread</a>, but neither are satisfactory to me. </li> <li> In general, PureBasic doesn't allow expressions, statements, or function calls to be split across multiple lines. There is also no line continuation operator; this was apparently <a href="http://forums.purebasic.com/english/viewtopic.php?p=33479&sid=3a090e47b61e8fd20e032007c24d7a32#p33479">ruled out 20 years ago</a>. </li> </ul> These all seem like reasonable features for an established, easy-to-use language to have. I don't understand why they are absent in PureBasic. </p> <p> Desite its flaws, I don't mind PureBasic. It seems somewhat stagnant, but it's charming in its own way. It might be more appealing if I were developing commercial software, but otherwise the commercial license is a deal breaker for me. </p> <h3>What I Like</h3> <ul> <li>Simple, intuitive, and easy to learn</li> </ul> <h3>What I Don't Like</h3> <ul> <li>Not Free Software</li> <li>Small development team and small community</li> <li>Missing features</li> <li>No support for the BSDs</li> </ul> <h3>Links</h3> <ul> <li><a href="https://www.purebasic.com">PureBasic Website</a></li> <li><a href="https://www.purebasic.fr/blog/">PureBasic Blog</a></li> <li><a href="https://www.purebasic.fr/english/">PureBasic Forum</a></li> <li><a href="https://freeshell.de/~luis/purebasic/about/index.php">PureBasic Review - Luis</a></li> <li><a href="https://schipplock.de/docs/pb.html">PureBasic Review - Andreas Shipplock</a></li> <li><a href="https://www.youtube.com/channel/UCseEtp7WLr7pfj4w-MDbhDw">Pure Programming video series - Guillaume</a></li> </ul> <h2 id="freebasic">FreeBASIC</h2> <p> <a href="https://www.freebasic.net/">FreeBASIC</a> is a close competitor to PureBasic. As implied by the name, FreeBASIC is Free Software&mdash;a welcome change. Even better is that it supports MS-DOS, which I had overlooked during my previous survey of DOS-compatible programming languages. If FreeBASIC can deliver the best parts of PureBasic, but with the advantage of DOS support and a Free Software license, then it would be a serious contender for my next language. To find out, I set to work rewriting my PureBasic Wordle clone in FreeBASIC. </p> <p> Like PureBasic, FreeBASIC is immediately intuitive. Its syntax is very similar to Visual Basic (both are descendents of QuickBasic). The documentation is of a fairly high quality, and more detailed than PureBasic's. Where it falls a little flat for me is the inconsistency between how different runtime library features are implemented. Many of them are implemented as methods (subroutines or functions). However, others are statements. For example, files are opened for reading with a statement like this: <figure> <figcaption class="code">Opening a file in FreeBASIC</figcaption> <pre class="code">Dim fileNum As Integer = FreeFile Open "filename" For Binary Access Read As #fileNum</pre> </figure> I suspect FreeBASIC does this to maintain compatibility with QuickBasic. Personally, I prefer the method syntax used by PureBasic and most other languages. For example: <figure> <figcaption class="code">Opening a file in PureBasic</figcaption> <pre class="code">Dim fileNum = ReadFile(#PB_Any, "filename")</pre> </figure> </p> <p> The main downside of FreeBASIC is debugging. FreeBASIC uses <a href="https://en.wikipedia.org/wiki/GNU_Debugger">GDB</a>. There is no official IDE with integrated debugger. Although you <i>can</i> <a href="https://www.freebasic.net/forum/viewtopic.php?t=29117">configure VSCode</a> to debug FreeBASIC code with GDB, I wasn't able to get this to work without breaking my app's console features. Even when debugging with an external terminal, colors wouldn't work properly and some key presses wouldn't get handled. I'm not sure why this happens&mdash;I didn't encounter this when debugging Replicalc and other C projects with VSCode and GDB. It's possible that I've just overlooked or misconfigured something, but it makes debugging tedious. </p> <p> Other than that, I haven't encountered any significant issues or annoyances. For my purposes, FreeBASIC seems like an ok alternative to C, but I would need to get GDB and VSCode co-operating before I use it seriously. </p> <h3>What I Like</h3> <ul> <li>Free Software license</li> <li>Solid documentation</li> <li>MS-DOS support</li> </ul> <h3>What I Don't Like</h3> <ul> <li>Issues debugging with VSCode</li> <li>Small community</li> </ul> <h3>Links</h3> <ul> <li><a href="https://freebasic.net">FreeBASIC Website</a></li> <li><a href="https://www.freebasic.net/wiki/DocToc">FreeBASIC Manual</a></li> </ul> <h2 id="pascal">Pascal</h2> <p> I came across Pascal a few times during my previous survey for MS-DOS programming languages. The classic Turbo Pascal compiler is now freeware, and <a href="https://www.freepascal.org/">Free Pascal</a> is Free Software. <a href="https://www.embarcadero.com/products/delphi">Delphi</a> is still available (and still obscenely expensive). However, I really don't enjoy Pascal syntax, and the community is tiny&mdash;maybe even smaller than what's left of the BASIC community. Pascal is a non-starter for me. </p> <h3>Links</h3> <ul> <li><a href="https://www.freepascal.org/">Free Pascal</a></li> <li><a href="https://www.lazarus-ide.org/">Lazarus IDE</a></li> <li><a href="https://castle-engine.io/modern_pascal_introduction.html">Modern Object Pascal Introduction for Programmers</a></li> <li><a href="https://dubst3pp4.github.io/post/2017-10-03-why-i-use-object-pascal/">Why I use Object Pascal - Marc Hanisch</a></li> <li><a href="http://blarg.ca/2018/10/14/turbo-pascal/">Turbo Pascal - Gered King</a></li> </ul> <h2 id="nim">Nim</h2> <p> <a href="https://nim-lang.org">Nim</a> looked like another good candidate. Its syntax is heavily inspired by Python, which is refreshing in a compiled language. The documentation is OK; it could be more detailed and better organised, but Nim is still relatively young. </p> <p> Unfortunately, like FreeBASIC, I encountered issues with debugging in VSCode. This time it <i>kind of</i> worked&mdash;I could at least pause on breakpoints. However, the names of local variables would be garbled, and stepping over my code would inexplicably step into the Nim standard library instead. I tried following guides such as <a href="https://github.com/jasonprogrammer/nim-debug-example">Jason Jones'</a>, but didn't have any luck fixing these issues. I would need to get debugging working properly in VSCode to give Nim a serious look. </p> <h3>What I Like</h3> <ul> <li>Free Software license</li> <li>Python-inspired syntax</li> </ul> <h3>What I Don't Like</h3> <ul> <li>Issues debugging with VSCode</li> </ul> <h3>Links</h3> <ul> <li><a href="https://freebasic.net">FreeBASIC Website</a></li> <li><a href="https://www.freebasic.net/wiki/DocToc">FreeBASIC Manual</a></li> </ul> <h2 id="go">Go</h2> <p> <a href="https://go.dev">Go</a> was designed at Google for productivity, simplicity, and concurrency. Concurrency isn't very important for my personal projects, but productivity and simplicity certainly are. </p> <p> After spending some time going through <a href="https://go.dev/tour/welcome/1">the Tour</a> and building some test projects, I was really impressed. Go is dead-simple; the syntax makes sense and actively improves on the conventional C style; and the VSCode tooling and debugging just works. In addition, I didn't encounter any significant shortcomings. Go is almost a simpler, garbage collected C. Maybe it's not so surprising considering Ken Thompson co-designed them both. </p> <p> My only major criticism of Go is that I don't like <a href="https://go.dev/blog/error-handling-and-go">its error handling</a>. In Go, functions can return multiple values (also not a fan), so the final value is often used to indicate whether an error has occurred. The main problem is that you can't nest function calls with multiple return values unless the outer function accepts multiple arguments of the same type. Instead, you're forced to call the two functions separately. I much prefer the C way of doing things, where you check for an error by passing a pointer to an integer. While you <i>can</i> do this in Go, it's not idiomatic and has issues due to Go's unique language features. It's not a big deal at the end of the day, so I'll just get used to it. </p> <p> I've now rewritten <a href="https://dheinemann.com/projects/replicalc">Replicalc</a> in Go. Replicalc isn't complex by any stretch, but this has been a good project to familiarise myself with the language, and it's now in a much better state for further development. </p> <h3>What I Like</h3> <ul> <li>Simple, intuitive, and easy to learn</li> <li>Large community</li> <li>Decent documentation</li> </ul> <h3>What I Don't Like</h3> <ul> <li>Idiomatic error handling uses multiple returns</li> </ul> <ul> </ul> <h3>Links</h3> <ul> <li><a href="https://go.dev">Go Website</a></li> <li><a href="https://go.dev/tour/welcome/1">A Tour of Go</a></li> <li><a href="https://pkg.go.dev/std">Standard Library documentation</a></li> <li><a href="https://go.dev/ref/spec">Language Specification</a></li> <li><a href="https://bitfieldconsulting.com/golang/rust-vs-go">Rust vs Go - Bitfield Consulting</a></li> <li><a href="https://kristoff.it/blog/why-go-and-not-rust/">Why Go and not Rust? - Loris Cro</a></li> </ul> <h2 id="conclusion">Conclusion</h2> <p> Overall, I'm extremely happy with Go. It's almost exactly what I've been looking for: simple, high-level, and compiles to native. It's definitely going to be my go-to language for personal projects. </p> <p> Rust has a tonne of potential. It would be my next choice after Go, and I'll likely use it in any projects where a garbage collector is unsuitable. If I could get past its syntax, Zig might be a good alternative; maybe it will grow on me. </p> <p> PureBasic has a lot of charm, but its commercial license is a deal-braker. Nim and FreeBASIC are both superior, but I wouldn't consider using them unless I can get debugging to work in VSCode. And either way, I don't think they have much to offer me that Go or Rust don't. </p> <p> Pascal is right out. </p>I Redesigned the Site Theme2022-04-09T00:00:00+10:002022-04-09T00:00:00+10:00https://dheinemann.com/posts/2022-04-09-redesigned-site-theme<p> On March 19th, I decided to overhaul my website theme. I loved the Windows 3.11-inspired layout and didn't want to get rid of it completely, but I wanted something a bit simpler and less graphical; I'm no artist, and I didn't want to be designing new icons for every link I add to the navigation bar. So I decided to do some cleaning up! </p> <p> Here are the results. </p> <div style="float: left; width: 100%"> <figure style="float: left;"> <a href="https://dheinemann.com/oldthemes/win311original/index.html"> <img src="https://dheinemann.com/assets/beforewin311change.png" alt="The original Windows 3.11 theme" width="640" height="480" style="border: 1px solid #000000"/> <figcaption>Before</figcaption> </a> </figure> <figure style="float: left;"> <img src="https://dheinemann.com/assets/afterwin311change.png" alt="The new Windows 3.11 theme" width="640" height="480" style="border: 1px solid #000000"/> <figcaption>After</figcaption> </figure> </div> <p> I'm really happy with the new theme. It preserves the Windows 3.11 style without being too overbearing. You can view an archive of the original theme's homepage <a href="https://dheinemann.com/oldthemes/win311original/index.html">here</a>. </p>How my RuneScape Account got Stolen—and how I got it Back2022-04-02T00:00:00+10:002022-04-02T00:00:00+10:00https://dheinemann.com/posts/2022-04-02-how-my-runescape-account-got-stolen<p> On Wednesday 30th March 2022, my RuneScape account was stolen. Getting it back has been&hellip; significantly more difficult than it should have been. </p> <h2>Stolen Account</h2> <p> When I woke up in the morning, there was an email waiting in my inbox. "We have received a request to reset the password for your account", it said. "To reset your password, please use the button below". </p> <p> RuneScape phishing emails often get through my spam filter, but this one looked authentic. Just in case, I tried logging into the game; it said my credentials were wrong. I double-checked them and tried again&mdash;no luck. My account had indeed been stolen. </p> <h2>How did this Happen?</h2> <p> My first thought was that my email account must have been compromised&mdash;how else could somebody have used the RuneScape password reset form to steal my game account? But my email account has 2FA enabled, and the access logs show no activity from IP addresses I don't recognise. </p> <p> My RuneScape account also had 2FA. Did the perpetrator have my TOTP key? Had they gotten around 2FA somehow? Could my computer be compromised? Maybe I had been keylogged? But if so, why go for my RuneScape account of all things? And besides, I'm not in the habit of installing untrusted software, and I run Linux&mdash;which I would not expect to be a typical target. </p> <p> I eventually figured out that my account had been stolen through the RuneScape Account Recovery process&mdash;the exact same process that I had to use to get it back. In order to submit an Account Recovery request, you must first attempt a password reset, which gets sent to your email address; this explains the email I received earlier, and why there were no suspicious login attempts. It's not <i>the</i> account takeover method, it's just a byproduct of it. It's a good thing though, otherwise I might not have been aware my account had been stolen. Sadly, the 2FA on my account counted for nothing since it was automatically removed by the Account Recovery process. </p> <h2>The Account Recovery Process</h2> <p> When I began the process of recovering my RuneScape account, I assumed I would be able to talk to a real person and prove my identity using my government ID or something. The name and address on my ID match the payment details on my RuneScape account. Unfortunately, this wasn't possible. </p> <p> RuneScape's Account Recovery process is <i>really bad</i>. There is no way to talk to a real person. You can't simply email Jagex support&mdash;even if you're a paying customer. Your only option is to fill out the Account Recovery form and hope that your answers are good enough for whoever happens to read it. If they aren't, you will receive a canned response that only provides a few clues for your next attempt. </p> <div style="text-align: center;"> <figure> <a href="https://dheinemann.com/assets/runescapeaccountrecovery.jpg"> <img src="https://dheinemann.com/assets/runescapeaccountrecovery.jpg" alt="The RuneScape Account Recovery Form" style="max-width: 100%"/> </a> <figcaption> The RuneScape Account Recovery Form </figcaption> </figure> </div> <p> Everything wrong with this form boils down to a single huge problem: RuneScape <i>is really old</i>. My account is nearly as old as the game itself! </p> <p> I don't remember my old passwords&mdash;especially from 10+ years ago. Who does? Luckily I have a password manager these days, and it has a record of the most recent ones. </p> <p> I don't remember the answers to my recovery questions. I probably set them when I was a child, and Jagex <a href="https://support.runescape.com/hc/en-gb/articles/115005381989-Recovery-Questions">doesn't allow them to be changed</a>. Best I could do here was guess. </p> <p> I don't remember the month and year that my account was created. RuneScape didn't use an email address back then, so I have no email to commemorate that date. I ultimately had to guess by reading through the historic patch notes, and by luck I was just one month off. </p> <p> I also don't know the ISP that I used when I created the account&mdash;I didn't create it at home because I didn't have the Internet back then. </p> <p> If you specify <i>Credit Card</i> as your earliest membership payment type, you are asked to supply the last four digits of the credit card that was used. I was sure I no longer had a record of this. </p> <p> The <i>Other Comments</i> field allows for just 300 characters to provide any additional information. It's barely enough. </p> <h2>Beurocracy</h2> <p> I filled out the form as best I can before heading to work, and submitted it. Several hours later, I received my response: denied. Jagex wanted more details about past payments. In my rush to get to work, I had simply guessed them, but I would evidently need to be more accurate. </p> <p> For my second Account Recovery request, I spent the afternoon poring over my archives to find as much information as possible. I managed to find a very early RuneScape membership receipt <i>and</i> the last four digits of the credit card that was used. I was shocked I had these! I filled out my second Account Recovery request with as much detail as possible, submitted it, and went to bed confident that I had supplied enough information to prove my identity. Unfortunately, it still wasn't enough. Jagex rejected my request because they wanted more information about the circumstances in which the account was created. </p> <p> For my third Account Recovery request, I racked my brains trying to think of any other details I could possibly add. I added them, submitted the request, and again it was denied&mdash;for the exact same reason. At this point it seemed like a lost cause&mdash;I had supplied everything that was asked of me and it still <i>wasn't good enough</i>. It seemed like my only recourse would be to keep resubmitting the form and hope to eventually find a reviewer who would accept it. </p> <p> For my fourth Account Recovery request, I decided to try something different. The RuneScape Account Recovery form hadn't been working properly. It had JavaScript errors, so the <i>Add Another Password</i> button didn't work. Since I could only write one password in that section, I had been writing the others in the <i>Other Comments</i> section. This in turn limited the amount of receipt numbers I could write in that same field. I had submitted my previous requests using Firefox for mobile and desktop; this time I tried using Chrome. The JavaScript errors disappeared and the form worked properly! I filled out every single field, submitted the form and went to bed. The next morning I had a pleasant surprise: my fourth Account Recovery request had been accepted. </p> <p> In the end, it seems that the thief only stole the small amount of gold I had in the OSRS bank, plus a few other low-value items. They had attempted to remove my RuneScape 3 bank pin, but were unsuccessful since I recovered my account within a few days. Losing some gold is unfortunate, but I'm grateful to have the account back. It has a lot of sentimental value to me. </p> <div style="text-align: center;"> <figure> <img src="https://dheinemann.com/assets/runescapecharacter.jpg" alt="My RuneScape character" style="max-width: 100%"/> <figcaption> My RuneScape character </figcaption> </figure> </div> <h2>Theories</h2> <p> As we've seen, the Account Recovery form is quite long and detailed. How could somebody other than me possibly have enough details to steal my account through this form? After reading about the experiences of other players who found themselves in my situation, I have two theories. </p> <p> First, as mentioned, my account is <i>old</i>. When I was younger and didn't know any better, I probably re-used these passwords on other sites - perhaps even including RuneScape fan sites. My accounts on RuneScape fan sites were sure to include my RuneScape account name, since these used to be the same as your character name. And if you knew the age of my earliest RuneScape fan site accounts, you could probably guess the age of my RuneScape account. So if the database of one of these old fan sites were leaked on the Internet, it <i>might</i> be possible to scrounge together enough information to successfully complete the Account Recovery form. But if this were the case, then I'd have thought my personal knowledge would have been enough to recover the account on my first attempt. </p> <p> The second and more concerning possibility is that the thief knew the exact answers to each part of the Account Recovery form. In 2018, Jagex <a href="https://secure.runescape.com/m=news/an-important-announcement?oldschool=1">sacked an employee</a> for "gross misuse of moderator priveleges". This employee allegedly <a href="https://www.reddit.com/r/2007scape/comments/9iubm6/compilation_of_user_personal_data_breached_by_mod/">used their access to customer data</a> to steal from players through the Account Recovery system. I don't believe this particular person had anything to do with the theft of my account, but it appears that this scenario is a very real possibility. How can a rightful account owner's memory stack up against somebody who has access to all the answers needed for the Account Recovery form? Hopefully I'll never have to deal with this. </p> <p> As for why my account was targeted, I really don't know. My best guess is the first theory&mdash;maybe somebody got their hands on an old fansite database, and has been gradually working through it to steal as much as possible. Some accounts as old as mine have exclusive and extremely valuable items like party hats. Unfortunately for me, my account has no valuable items whatsoever; no max-level skills; no achievements. It doesn't even have a sought-after character name (three-letter names are common hacking targets). My account is entirely unremarkable. This makes me suspect that the thief didn't know any better, and simply stole it in hopes that it <i>might</i> have something useful. Somebody with access to the RuneScape account database would have known not to waste their time! </p> <div style="text-align: center;"> <figure> <a href="https://dheinemann.com/assets/runescapeaccountrecoveryresponse.jpg"> <img src="https://dheinemann.com/assets/runescapeaccountrecoveryresponse.jpg" alt="Sample Account Recovery Response" style="max-width: 100%"/> </a> <figcaption> A canned Account Recovery response </figcaption> </figure> </div> <h2>How to Protect Yourself</h2> <p> I'm relieved to have my account back, but what can you do to protect yours? Here are my recommendations. </p> <p> <b>Enable 2FA</b> on both your RuneScape account and your email account&mdash;including any previous email accounts that you still hold onto. If somebody gets access to your present or past email accounts, they will have access to critical details needed for the Account Recovery form. Protect these carefully! </p> <p> <b>Use a bank pin</b>. I thought this was unnecessary since I had 2FA enabled. Unfortunately, Jagex will simply remove 2FA if somebody hijacks your account through the Account Recovery system. However, they won't remove the bank pin! Using a bank pin will ensure that you have 3-7 days to recover your account before its bank can be raided. Don't forget that you need to set the bank pin individually in both OSRS <i>and</i> RuneScape 3. </p> <p> <b>Find out the age of your account</b>, and write it down somewhere safe so that you can use it in future Account Recoveries. To do this, log into OSRS and talk to Hans in the Lumbridge Castle courtyard. He will tell you precisely how old your account is, in days. You can use this information to find out the date that your account was created. </p> <p> <b>Keep a record</b> of your old passwords and payment receipts so that you can use them in future Account Recoveries. </p> <p> <b>Use Google Chrome</b> when submitting your RuneScape Account Recovery form. I dislike Chrome, but it seems that the form doesn't work properly in Firefox. </p> <h2>Future</h2> <p> Incidentally, my account was stolen the day after Jagex announced <a href="https://secure.runescape.com/m=news/2022-account-services--player-security-improvements?oldschool=1">sweeping improvements</a> to account management and security. These changes are very welcome&mdash;if they had existed earlier, it wouldn't have been possible to steal my account like this. Hopefully Jagex follows through and gets them implemented by the end of the year. </p> <p> There's also this little snippet hidden at the end of the page. Maybe we'll be able to get help from real support staff in the future? </p> <blockquote> <b>Live Chat with Customer Service</b><br> Additionally, we are also investigating additional service enhancements. We are in the process of upgrading the Support Centre and will explore offering a form of live chat for some support queries to understand the challenges and benefits to players before we determine if this is something we can roll out to some or all players. </blockquote>Why I don't use Windows2022-03-23T00:00:00+10:002022-03-23T00:00:00+10:00https://dheinemann.com/posts/2022-03-23-why-i-dont-use-windows<p> I grew up on Windows and have used it throughout most of my life. However, Windows 8 marked the beginning of a long decline that has now culminated in the bloated mess that is Windows 11: a desktop operating system that spies on its users, forces unwanted apps and advertisements upon them, and increases e-waste through forced obsolescence of otherwise good hardware. </p> <h2>Telemetry (a.k.a. <i>Spying</i>)</h2> <p> Telemetry is the polite name for the data that software collects about you and your computer, and sends back home to its publisher. Windows has <i>a lot</i> of telemetry. Some of it can be disabled easily through the Settings app; a lot of it can't. </p> <p> Did you know that your Windows computer tells Microsoft which applications you have installed? I suspect most people don't, even though <a href="https://support.microsoft.com/en-us/windows/diagnostics-feedback-and-privacy-in-windows-28808a2b-a31b-dd73-dcd3-4559a5199319">Microsoft isn't exactly shy about it</a>. Your list of installed applications is part of the "Required" diagnostic data set. Microsoft doesn't allow this to be turned off through Settings, but it can still be disabled using <a href="https://windowsreport.com/disable-windows-11-telemetry/">a guide</a> if you're a little tech savvy. Guides like this have become increasingly essential as Windows' telemetry has grown over the years. Unfortunately, these guides aren't a permanent solution because Microsoft likes to deploy new telemetry through Windows Updates (which are both mandatory and automatic). Keeping up with it is a never-ending game of Whack-a-Mole. </p> <p> But perhaps worse than telemetry is the idea of connecting your Windows computer to your Microsoft account. Microsoft began encouraging users to do this in Windows 8 by presenting it as an option during installation. Then in Windows 8.1, they tried to trick users into thinking it was the <i>only</i> option by hiding the alternative at the bottom a different form. Now in Windows 11, using a Microsoft account <i>actually is</i> the only option. If you don't want your operating system connected to Microsoft's cloud, you're out of luck. </p> <h2>Unwanted Apps and Ads</h2> <p> Windows has always had a few non-essential extras, whether it's games like Solitaire or redundant tools like Windows Messenger. However, Windows 8 began the tradition of installing outright trash like <a href="https://blogs.bing.com/search/2012/08/15/introducing-bing-on-windows-8/">Bing Travel</a>. We're at the point now where it's considered <i>totally normal</i> to waste time uninstalling unwanted apps from a fresh Windows installation. And even then you aren't safe because, once again, Microsoft can deploy more unwanted apps through Windows Updates&mdash;including ones you previously removed. </p> <p> Then there's the advertising. As <a href="https://den.dev/blog/windows-priority-shuffle/">recounted by Dan Delimarsky</a>, we've seen: </p> <ul> <li><a href="https://twitter.com/teroalhonen/status/786619324819791872">Ads in Windows Explorer</a></li> <li><a href="https://www.bleepingcomputer.com/news/microsoft/windows-10-start-menu-promo-for-microsoft-edge-cant-be-disabled/">Ads in the Start Menu</a></li> <li><a href="https://www.bleepingcomputer.com/news/microsoft/microsoft-breaks-windows-11-start-menu-taskbar-with-teams-promo/">Ads in the Start Bar</a></li> <li><a href="https://www.bleepingcomputer.com/news/microsoft/microsoft-tests-office-ads-in-windows-10-wordpad/">Ads in WordPad</a></li> <li><a href="https://techcommunity.microsoft.com/t5/articles/introducing-buy-now-pay-later-in-microsoft-edge/m-p/2967030">Ads in Edge</a></li> </ul> <p> And let's not forget about <a href="https://www.howtogeek.com/243263/how-to-disable-ads-on-your-windows-10-lock-screen/">ads in the Lock Screen</a>! Imagine paying for a product that shows you unwanted advertisements&hellip; </p> <div style="text-align: center;"> <figure> <img src="https://dheinemann.com/assets/mixedreality.jpg" alt="Mixed Reality Portal" style="max-width: 100%"/> <figcaption> Preinstalled VR headset software? I can't wait to use this with the VR headset I don't have ¯\_(ツ)_/¯ </figcaption> </figure> </div> <h2>Forced Obsolescence</h2> <p> Lastly, there's the forced obsolescence. Beginning with Windows 11, an 8th-generation CPU (or newer) is required&hellip; despite the fact that 7th-generation CPUs were still on the market less than a year before its release. Now these old CPUs are just about worthless because they have no upgrade path. The average consumer doesn't know how to upgrade computer parts and isn't going to bother. I wouldn't blame them. </p> <p> Microsoft states that this CPU requirement is for <i>security reasons</i>, presumably so that BitLocker can be enabled out-of-the-box. Full-Disk Encryption is great&mdash;but it doesn't warrant dropping support for older CPUs when they could simply be grandfathered in without BitLocker. The end result is that most consumers will have to throw their old computer away in order to upgrade to Windows 11 when, realistically, it would run well enough on even a 4th-generation CPU. </p> <h2>Conclusion</h2> <p> Microsoft can get away with these consumer-hostile measures because they have a captive audience. Alternatives like Linux and MacOS simply aren't the same, and very few Windows users would ever consider switching; Microsoft can largely do whatever they want. It seems like Windows' best days are long behind it now, and based on current trends, I don't see it improving any time soon. Even if it did, I don't expect I'll go back. You can pay me to use Windows, but I won't do it for free. </p>My History with Linux2022-03-15T00:00:00+10:002022-03-15T00:00:00+10:00https://dheinemann.com/posts/2022-03-15-my-history-with-linux<p> I've long held an interest in Linux. I didn't have access to the Internet until I was about 12 years old. For a long time, Linux was this mythical alternative operating system outside of my grasp. I knew it existed, but didn't have the resources to experience it for myself. </p> <p> Then one day in 2002, PC User magazine came bundled with a copy of <a href="https://distrowatch.com/table.php?distribution=lycoris">Lycoris Desktop L/X</a>, an obscure newbie-friendly distro in the vein of Mandrake. I remember being blown away by the fact that it let you play solitaire during installation (something I have never seen since). However, this was soon met with disappointment. Lycoris Desktop L/X came with a fairly basic set of packages; without Internet access, there wasn't much to do besides play around with <a href="https://games.kde.org/games/ktuberling/">KTuberling</a>, and I soon went back to Windows 98 SE. </p> <p> A few years later, I saw a copy of <a href="https://lwn.net/Articles/139887/">Debian Sarge</a> at the news agency, complete with a 100-page book explaining everything from installation to general usage and games&mdash;all written for newbies like me. I think it must have come on multiple CDs because I remember it including thousands of packages. This was perfect for me because I was limited to 56k dial-up Internet at the time. I had a blast trying out all the games. <a href="http://fillets.sourceforge.net/">Fish Fillets</a> was my favourite. </p> <p> Eventually I got ADSL and was able to start exploring Linux distros for myself. I think maybe <a href="https://web.archive.org/web/20070422042629/https://ubuntu.com/getubuntu/releasenotes/704tour">Ubuntu 7.04 (Feisty Fawn)</a> was the first of these. For a long time I used <a href="https://en.wikipedia.org/wiki/CrunchBang_Linux">CrunchBang</a>. Other than short stints <a href="https://dheinemann.com/posts/2011-07-03-attack-of-the-slack"> with Slackware</a>, Arch Linux, and OpenBSD, I've largely stuck with Debian and Debian-based distros, perhaps out of familiarity. </p> <p> These days I'm using <a href="https://getfedora.org">Fedora</a>. I was using Debian beforehand, but it's always a <i>little bit</i> janky. Fedora has been perfect from the start for me, and I don't see myself changing distros any time soon. It has a high level of polish, and packages receive frequent updates despite not being a rolling release distro. For me, this achieves the perfect balance of reliability, and up-to-date packages. I recommend giving Fedora a shot! </p>.NET Value Types and Reference Types Explained2022-02-14T00:00:00+10:002022-02-14T00:00:00+10:00https://dheinemann.com/posts/2022-02-14-dotnet-value-types-and-reference-types-explained<p> If you're using C# or a similar programming language, you may have heard the terms "value type" and "reference type". What are they? How are they different from one-another? Let's find out. </p> <p> This article is written from a C# perspective, but it applies to all .NET languages (Visual Basic, PowerShell, etc.), and likely also similar languages such as Java. </p> <h2>Defining Value Types and Reference Types</h2> <p> In C# and other .NET languages, data types fall into two categories: value types, and reference types. Value types include the built-in <a href="https://docs.microsoft.com/en-us/dotnet/standard/numerics">numeric types</a> (int, long, double, etc), and <a href="https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/struct">structs</a> such as <a href="https://docs.microsoft.com/en-us/dotnet/api/system.datetime"><code>DateTime</code></a>. Reference types include <a href="https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/interface">interfaces</a>, and objects such as <a href="https://docs.microsoft.com/en-us/dotnet/api/system.string"><code>string</code></a>, <a href="https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.list-1"><code>List&lt;T&gt;</code></a>, and <a href="https://docs.microsoft.com/en-us/dotnet/api/system.array">arrays</a>. </p> <p> Value types and reference types are defined by how their variables work. A value type variable always contains the value itself. If I set an integer variable to 5, then it contains the value 5. On the other hand, a reference type <i>doesn't</i> contain the value. Instead, it contains a reference to it&mdash;a pointer to its address in memory. If I create a variable for a string, that variable actually contains an integer representing the string's memory address. </p> <p> For the most part, C# and similar languages will automatically handle references so that you don't need to deal with them directly. However, it's important to understand how value types and reference types behave, because misunderstanding the nuances can lead to unexpected bugs. </p> <h2>Value Types</h2> <p> As explained, variables of value types always contain the value itself. If I create an integer variable and set it to 5, then it actually contains the number 5. <figure> <pre class="code">int foo = 5;</pre> </figure> </p> <p> If I copy this variable into a separate variable, the value itself is copied. I can then change the copied variable without affecting the original: <figure> <pre class="code">int foo = 5; int bar = foo; bar = 10; Console.WriteLine(foo); // prints 5 Console.WriteLine(bar); // prints 10</pre> </figure> </p> <p> Similarly, if I pass this variable to a function, then its value gets copied to that function. The function can modify its copy of the value without affecting the original: <figure> <pre class="code">public static void ChangeTo10(int n) { n = 10; } public static void main() { int foo = 5; ChangeTo10(foo); Console.WriteLine(foo); // prints 5 }</pre> </figure> </p> <p> This behaviour is known as "passing by value"&mdash;copying a value type variable copies the value itself. </p> <h2>Reference Types</h2> <p> Reference types work differently. As explained, variables of reference types contain a reference to the value&mdash;not the value itself. Let's create a new class named <code>Person</code> to demonstrate: <figure> <pre class="code">public class Person { public string Name { get; set; } public Person(string name) { Name = name; } }</pre> </figure> </p> <p> If I define a variable and initialize it to a new instance of the <code>Person</code> class, then this variable contains a reference to the object's location in memory: <figure> <pre class="code">public static void main() { // person contains a reference to the object, not the object itself. var person = new Person("John"); }</pre> </figure> </p> <p> If I then copy this variable to a second variable, the reference gets copied, but the value does not. This means that both variables now point to the exact same object in memory. This can be tested using <a href="https://docs.microsoft.com/en-us/dotnet/api/system.object.referenceequals"><code>Object.ReferenceEquals()</code></a>: <figure> <pre class="code">public static void main() { var person1 = new Person("John"); var person2 = person1; Console.WriteLine(object.ReferenceEquals(person1, person2)); // prints True }</pre> </figure> </p> <p> What does this mean? It means that if two variables point to the same object, modifying one will affect the other: <figure> <pre class="code">public static void main() { var person1 = new Person("John"); var person2 = person1; person2.Name = "Jane"); Console.WriteLine(person1.Name); // prints "Jane" }</pre> </figure> </p> <p> This also applies when passing a variable as a function argument: <figure> <pre class="code">public static void ChangeToJane(Person p) { p.Name = "Jane"; } public static void main() { var person1 = new Person("John"); ChangeToJane(person1); Console.WriteLine(person1.Name); // prints "Jane" }</pre> </figure> </p> <p> This behaviour is known as "passing by reference"&mdash;copying a reference type variable copies the reference, not the value itself. </p> <h2>ByRef, ref, and [ref]</h2> <p> But if reference types are always passed by reference, then what are the <a href="https://docs.microsoft.com/en-us/dotnet/visual-basic/language-reference/modifiers/byref"><code>ByRef</code></a> (VB.NET), <a href="https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/ref"><code>ref</code></a> (C#), and <a href="https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_ref"><code>[ref]</code></a> (PowerShell) keywords for? </p> <p> These keywords allow you to pass a reference type to a function in such a way that a reference to the original variable is passed rather than the reference to the object. This allows the function to change which object the original variable points to in memory: <figure> <pre class="code">public static void ChangeToJaneByReference(ref Person p) // note the ref keyword { p = new Person("Jane"); } public static void main() { var person1 = new Person("John"); // person1 points to the Person John ChangeToJaneByReference(ref person1); // person1 now points to the Person Jane Console.WriteLine(person1.Name); // prints "Jane" }</pre> </figure> </p> <p> This is impossible without the <code>ref</code> keyword: <figure> <pre class="code">public static void ChangeToJane(Person p) { p = new Person("Jane"); } public static void main() { var person1 = new Person("John"); // person1 points to the Person John ChangeToJane(person1); // person1 still points to the Person John because ChangeToJane() cannot // change what person1 points to without using the ref keyword Console.WriteLine(person1.Name); // prints "John" }</pre> </figure> </p> <p> What about value type parameters? <code>ref</code> works for them too! It allows a function to modify the value contained in a variable passed to the function in the same way that <code>ref</code> works for reference types: <figure> <pre class="code">public static void ChangeTo10ByReference(ref int n) // note the ref keyword { // Since n represents the memory address of whatever variable was passed // in, modifying it changes the value of that variable. n = 10; } public static void main() { int foo = 5; ChangeTo10ByReference(ref foo); Console.WriteLine(foo); // prints 10 }</pre> </figure> </p> <p> Enabling an original variable to be modified in this way is the only purpose of the <code>ref</code> keyword. If your method uses the <code>ref</code> keyword but doesn't change the value or reference of a variable passed to it, then the <code>ref</code> keyword serves no purpose. </p> <p> Additionally, using <code>ref</code> should generally be avoided unless there's a really good reason why a function cannot simply return a modified object using a <code>return</code> statement. Using the <code>return</code> statement is simpler and easier to understand. </p> <h2>Why do Value Types and Reference Types Work this Way?</h2> <p> Hopefully that clears up the differences between value types and reference types, but you may be left wondering why things work this way. Wouldn't it be simpler if all values could simply be passed by value? Maybe. But it isn't practical. </p> <p> While value types are generally small in size, objects can be massive. If you passed a string representing a very large text file to a function, you probably wouldn't want the contents of that string to be duplicated in-memory. If you didn't specifically need the string to be copied, then it would be both a waste of memory and CPU cycles. And the problem would be even worse if that function then passed the string to <i>another</i> function, and so-on&hellip; It's far better for objects to be passed by reference by default. If a programmer has a specific need to duplicate a string or other object, they can do it themselves. </p> <h2>Tying it All Together</h2> <p> How does this affect your code? It means that value types and reference types need to be handled in fundamentally different ways. For example: <ul> <li> If you want to pass a value type to a function so that it can be modified, that function needs to return the modified value (or it could use the <code>ref</code> keyword, but this should be avoided). </li> <li> Be aware that if you write a function that modifies a reference type parameter, it will modify the original object that the calling function passed in. </li> <li> If your function modifies a reference type parameter, then it isn't necessary for it to also return that parameter (however, this can still be useful in some cases, such as when building a <a href="https://en.wikipedia.org/wiki/Fluent_interface">fluent interface</a>). </li> <li> If your function doesn't change a parameter's original variable to a different value or object, then it doesn't need to use the <code>ByRef</code>, <code>ref</code>, or <code>[ref]</code> keywords. </li> </ul> </p>PowerShell Gotchas2022-02-13T00:00:00+10:002022-02-13T00:00:00+10:00https://dheinemann.com/posts/2022-02-13-powershell-gotchas<p> PowerShell is an extremely *<i>ahem</i>* powerful programming language. Since 2017, it has been my language of choice whenever it comes to scripting and automation on Windows, including small-scale systems integrations. </p> <p>PowerShell has two key benefits over the alternatives: <ul> <li> <b>.NET integration:</b> PowerShell has direct access to the whole .NET class library. If you already know .NET (for example, from a C# background), you will find this immensely helpful. </li> <li> <b>First-class support:</b> Microsoft is investing heavily into PowerShell as the successor to VBScript and Batch scripts. Almost any Windows administration task can be automated through PowerShell&hellip; not to mention Exchange Online, etc. If you're a Windows system administrator and still using VBScript and Batch, you're living in the ancient past<sup>&dagger;</sup>. </li> </ul> </p> <p> But PowerShell isn't without its warts. Notably, it suffered from an awful OOP implementation for <i>10 years</i> before classes were added in PowerShell 5. Beyond that, it has a number of quirks and gotchas&mdash;unexpected behaviours that can take you by surprise if you aren't aware of them, especially if you are used to other .NET languages such as C#. </p> <p> Here are a few things to watch out for when working with PowerShell. </p> <h2>By Default, String Comparisons are Case-Insensitive</h2> <p> PowerShell is case-insensitive by default: <figure> <pre class="code">"foo" -eq "FOO" # prints True</pre> </figure> </p> <p> To perform a case-sensitive comparison, case-sensitive comparison operators must be used. For example: <figure> <pre class="code">"foo" -ceq "FOO" # prints False</pre> </figure> Refer to <a href="https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_comparison_operators">about_Comparison_Operators</a> for a list of available operators. </p> <h2>$null Comparisons can be Inconsistent</h2> <p> If you write your <code>$null</code> comparisons with <code>$null</code> on the right-hand side (as is common with most programming languages), you may encounter inconsistent results. For example: <figure> <pre class="code"># Based on the sample code from # https://github.com/PowerShell/PSScriptAnalyzer/blob/development/RuleDocumentation/PossibleIncorrectComparisonWithNull.md if (@() -eq $null) { "True" } else { "False" } # prints False, as expected if (@() -ne $null) { "True" } else { "False" } # also prints False ???</pre> </figure> This is due to differences in how the comparison operation operators work for scalars (individual values) versus arrays. A detailed explanation can be found <a href="https://github.com/PowerShell/PSScriptAnalyzer/blob/development/RuleDocumentation/PossibleIncorrectComparisonWithNull.md"> in the PSScriptAnalyzer documentation</a>. </p> <p> To avoid this problem, consider placing <code>$null</code> on the left-hand side during comparisons. This way, <code>$null</code> (a scalar) is the subject of the comparison, and the behaviour will be consistent regardless of whether it's compared to another scalar, or to an array: <figure> <pre class="code">if ($null -eq @()) { "True" } else { "False" } # prints False if ($null -ne @()) { "True" } else { "False" } # prints True</pre> </figure> </p> <h2>By Default, PowerShell Continues after Non-Fatal Errors</h2> <p> By default, if PowerShell encounters a non-fatal error, it will automatically continue execution. This can be undesirable for scripts and automations, where you generally want to stop if an unexpected error occurs. </p> <p> If you don't want PowerShell to blindly continue after a non-fatal error occurs, you have three options: <ol> <li> Use try/catch blocks. A non-fatal error is treated as an exception for the purpose of try/catch blocks, and can be handled however you please. </li> <li> Use the <a href="https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_commonparameters?view=powershell-7.2#-erroraction"> <code>-ErrorAction</code></a> parameter on individual cmdlets. This can be used to cause the script to stop on a per-cmdlet basis. </li> <li> Use <a href="https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_preference_variables?view=powershell-7.2#erroractionpreference"> <code>$ErrorActionPreference</code></a>, which sets PowerShell's default behaviour when a non-fatal error occurs. </li> </ol> </p> <p> For example, to make a PowerShell script automatically stop when an unhandled non-fatal error occurs, set <code>$ErrorActionPreference</code> to Stop: <figure> <pre class="code">$ErrorActionPreference = Stop</pre> </figure> </p> <p> Note that <code>-ErrorAction</code> and <code>$ErrorActionPreference</code> do not affect the handling of errors caught in a try/catch block; they will still catch and handle errors regardless of the <code>-ErrorAction</code> parameter and <code>$ErrorActionPreference</code> setting. </p> <h2>Adding Array Elements with += Recreates the Array</h2> <p> If you've been using PowerShell for a while, you may be accustomed to adding elements to an array using the <code>+=</code> operator: <figure> <pre class="code">[int[]] $numbers = @() for ($i = 1; $i -le 10; $i++) { $numbers += $i }</pre> </figure> </p> <p> However, the array <code>+=</code> operator comes at a significant performance cost because it recreates the array every time. As explained in <a href="https://docs.microsoft.com/en-au/powershell/module/microsoft.powershell.core/about/about_arrays?view=powershell-7.2#manipulating-an-array">the documentation</a>: <blockquote> When you use the <code>+=</code> operator, PowerShell actually creates a new array with the values of the original array and the added value. This might cause performance issues if the operation is repeated several times or the size of the array is too big. </blockquote> </p> <p> If you need to grow an array inside of a loop, consider using the .NET <a href="https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.list-1">List&lt;T&gt;</a> class instead. New elements can be added to a <code>List&lt;T&gt;</code> without having to recreate an array every time: <figure> <pre class="code">[System.Collections.Generic.List[int]] $numbers = [System.Collections.Generic.List[int]]::new() for ($i = 1; $i -le 10; $i++) { $numbers.Add($i) }</pre> </figure> </p> <p> So why does PowerShell need to clone the array when the <code>+=</code> operator is used? Like many programming languages, PowerShell allocates a single contiguous block of memory for arrays and their elements. All elements of an array are located next to each other in memory, arranged in order of their index (index 0 being the first element, index 1 being the second, and so on). </p> <p> Since array memory is allocated as a contiguous block, it isn't possible to expand the array because the memory immediately following it could be in-use by something else. Instead, PowerShell finds a new contiguous block of memory large enough to store the original array plus the new element, allocates it, copies the elements across, then de-allocates the original array&mdash;a task that can be very slow when performed frequently, such as inside of a loop. </p> <p> Internally, the .NET <code>List&lt;T&gt;</code> class also uses an array to store its elements. So how does it avoid the performance penalty of recreating the array whenever an element is added? Simple: by allocating space for <a href="https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.list-1.capacity">extra array elements</a> beyond what's initially required. By maintaining a buffer or set of spare array elements, new data can be added to the <code>List&lt;T&gt;</code> without having to recreate the array every time. This is why adding new values to a <code>List&lt;T&gt;</code> is significantly faster than adding new elements to an array. </p> <h2>By Default, Accessing Uninitialized Variables is not an Error</h2> <p> By default, uninitialized variables are considered equivalent to <code>$null</code>, and referencing them is not considered an error: <figure> <pre class="code">$null -eq $thisVariableDoesNotExist # returns $true</pre> </figure> </p> <p> This can cause all kinds of unexpected issues. For example, it makes it easier for typos to go unnoticed: <figure> <pre class="code">[bool] $foo = $true if ($fo) { # $fo evaluates to $false because it's uninitialized, so this block never # gets executed. Write-Host "Foo" }</pre> </figure> </p> <p> To change this behaviour, use <a href="https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/set-strictmode?view=powershell-7.2"> <code>Set-StrictMode</code></a> to set strict mode to version 1.0 or greater. </p> <p> Similarly, exceeding the bounds of an array is also not considered an error by default. To change this behaviour, set strict mode to 3.0 or greater. </p> <h2>PowerShell Scoping is not Strictly Lexical</h2> <p> PowerShell's scoping isn't strictly lexical&mdash;variables declared inside of if statements and for loops can be accessed outside of them: <figure> <pre class="code">if ($true) { [string] $foo = "foo" } Write-Host $foo # prints 'foo'</pre> </figure> </p> <p> I prefer not to use this functionality because it can be confusing. Be wary of it during code reviews&mdash;what looks like a reference to an uninitialized variable may actually be valid. </p> <h2>Piping to Out-Null can be Slow</h2> <p> <code>Out-Null</code> can be used to discard the output of a cmdlet via the pipeline. However, using the pipeline adds a performance overhead. This overhead can become significant when piping a large number of cmdlets to <code>Out-Null</code> inside of a loop. Instead, consider assigning the cmdlet to <code>$null</code>, which is much faster: <figure> <pre class="code"># With Out-Null Get-ChildItem | Out-Null # With $null $null = Get-ChildItem</pre> </figure> </p> <h2>Conclusion</h2> <p> That covers all the gotchas I've encountered as a PowerShell developer so far. What kind of gotchas have you encountered? </p> <hr> <p class="footnote"> <sup>&dagger;</sup> Unless you're maintaining a pre-PowerShell version of Windows, in which case, you have bigger problems than PowerShell could ever solve. </p>Archiving a Website with Wget2022-04-09T00:00:00+10:002022-02-05T00:00:00+10:00https://dheinemann.com/posts/2022-02-05-archiving-a-website-with-wget <p> Web pages and even entire websites disappear every day, never to be seen again. Sometimes the author decides to take them offline. Sometimes they pass away. Sometimes (especially with free services), the hosting provider simply <a href="https://en.wikipedia.org/wiki/Yahoo!_GeoCities#Closure">purges everything to save a buck</a>. The good news is that you can do something about it, at least on a personal level. </p> <p> If you aren't too tech savvy, you can easily archive individual pages for free using the <a href="https://web.archive.org/">Wayback Machine</a>. Or if you're prepared to pay a very reasonable price for the privilege, <a href="https://pinboard.in">Pinboard</a> will archive all of your bookmarks for you. </p> <p> But what happens if you need to archive an entire website? Nobody wants to manually copy and paste individual URLs into an archiving service. If you're prepared to get your hands dirty, there's Wget. </p> <p> <a href="https://en.wikipedia.org/wiki/Wget"><b>Wget</b></a> is a command line tool for fetching content from web servers. Using the right command line arguments, it is very effective at downloading a whole website. Wget comes installed on most Linux distributions. Windows users will need to find a Windows build online such as the (now quite old) <a href="http://gnuwin32.sourceforge.net/packages/wget.htm">GnuWin32 packages</a>, or use <a href="https://cygwin.com">Cygwin</a>. </p> <h2>Archiving a Website</h2> <p> To archive a website with Wget, use: <figure> <pre class="code">wget -mpckE --user-agent="" -e robots=off --wait 1 www.foo.com</pre> </figure> </p> <b>Explanation</b> <p> <ul> <li> <b>-m (Mirror):</b> Turns on mirror-friendly settings like infinite recursion depth, timestamps, etc. </li> <li> <b>-p (Page Requisites):</b> Includes page dependencies such as images, style sheets, etc. in the download. </li> <li><b>-c (Continue):</b> Resumes a website that has been partially downloaded.</li> <li> <b>-k (Convert Links):</b> Convert absolute hyperlinks into relative hyperlinks for offline viewing. </li> <li> <b>-E (Adjust Extension):</b> Changes web page file extensions to <code>.html</code> for offline viewing. </li> <li> <b>-user-agent="":</b> Tells Wget not to identify itself. This can be useful if a website is known to detect and block Wget. You can also set your user agent to that of a web browser. </li> <li> <b>-e robots=off:</b> Tells Wget to ignore robots.txt, if it exists. robots.txt is used to restrict which pages web crawlers such as Wget, GoogleBot, etc. will access. </li> <li> <b>-wait 1:</b> Tells Wget to wait 1 second between each page or resource download. This avoids being unreasonably taxing on the servers. </li> </ul> </p> <p> The smallest websites only take a few minutes to download. Others can take significantly longer. When Wget is done, you'll have a folder structure matching the website that you downloaded. Open it up, and then open the index.html page to see the finished result. </p> <h2>Further Information</h2> <ul> <li><a href="https://www.gnu.org/software/wget/">Wget Official Website</a></li> <li><a href="https://linux.die.net/man/1/wget">Wget Man Page</a></li> </ul>