TrickJarrett.com

Integrating with Mastodon and the Fediverse

If this works correctly, the blog will identify this entry's link as being a post on the Fediverse which then properly generates code to make an embed on the blog. (Update: It worked perfectly.) The post is the first in a short thread, where I delved into this. I'm not going to embed it into the newsletter email, though I am beginning to think I should re-evaluate that.

First, I found a free and open API which tracks and provides info on instances of Fediverse. I don't have a need for the list to be perfect or completely up to date, so I just query for the 1000 biggest instances, once a week. The risk of problems is low enough I don't think it is worth me devoting more time to it until it becomes a problem.

So now, when I create a post on the blog, it checks the URL against the list of Instances and if it finds a match then it knows to generate it as a Fediverse embed. Nice and simple.

Apologies to the newsletter readers, there has been a bug where some post links were not included in the emails. I fixed that this morning.

The newsletter goes out most evenings and shares the entire day's posts for easy consumption in your inbox.

Did a little coding on Glowbug today. First in a long time. With the growth of Mastodon as a social network, decided I needed to improve the cron bot daily posts for Mastodon. On twitter, it self deletes the previous post before making a new one. I was working on getting that implemented for Mastodon, so my account will not be cluttered with the bot posts.

I'll know better tomorrow if it works or not, but it should. The API is very straightforward for Mastodon.

"The Sum of All Knowledge"

A story looking back at a man's father's striving for education out of nothing as a child coal miner, to the age of the Internet and how it shaped the son's life, but then it continues and criticizes the current state of the Internet - equating much of it to a burning library.

Coding for flexible scheduling

This morning's back-of-my-mind coding puzzle I'm trying to figure out. How to best handle flexible scheduling of things. I want to do a rotation of posts that are of a similar format: "What I'm Reading," "What I'm watching," "What I'm listening to," and "What I'm working on" - these would rotate through books, tv, podcasts / audiobooks, and the things I'm spending my time on.

But I'd like to add it as a feature in Glowbug as a scheduled prompt sort of thing. But I need to figure out how I store it flexibly so I can say "First Saturday", "Second Saturday", etc.

Right now my idea is to make use of PHP's strtotime() function which works on the above, but doesn't handle "every Monday" etc. So I'm trying to wrap my head around this puzzle and figure out how to tackle it.

Respond Via Email

I may live to regret this, but for now - given the modest traffic to this blog - I've added a "respond via email" link to every post in an effort to foster conversations and get feedback from readers.

Aug. 13, 2022 - Glowbug's Tag Synonyms

Transcript:

This morning has been nice and relaxed. Had a nice breakfast with family and then had to come back and do some "homework" as I have been tasked with teaching my niece and her son to play the Pokemon card game and I had to refresh on the rules.

After which I played a few games of chess. I lost most of them, so I switched to some coding. Glowbug now handles tag synonyms. If I rename a tag it saves the old tag info so that if I try to recreate that tag, it points to the synonym instead. Should help avoid some silly tag errors from occurring more than once. Hopefully.

Trick

I just remembered an idea I had for this blog. I have half-implemented a "Chapter" feature, which is an old blog feature I had from the days when I was primarily blogging as a public journal of my life. Something which I still do, but nowhere near as much.

The idea of the Chapters feature was to allow easy grouping of posts chronologically into eras of life. Like you could do monthly, or based on life events, etc.

But a problem with this idea is that it is primarily something done after-the-fact and doing it in the middle of experiencing life is difficult.

My realization this morning was also that I should implement "series" as the feature for the blog. Sure, some might be chapters of life updates, but it would also handle things like my post from a few days ago regarding basic online and personal security to make it more difficult to hack you. The next post is going to be about password managers, what they are, and why to use one. I'm not sure what the third post will be, but whatever it is, will go into that series.

I've just got to figure out how to add it to the admin UI and then I'll sit down and code it.

In a mood

I would expect today to be a light blogging day, at least from an outward link perspective.

I've woken up with a feeling of malaise and general blahness to the world. I was listening to the Offline podcast this morning and I found myself getting... angry. And not in a rational way. The guest is talking about the danger TikTok presents, something which has been discussed a great deal online, but I found the tone of the conversation and the angle taken to just chafe. Not that I am a TikTok proponent or fan, I use it, but my total usage in a week is probably on the order of... an hour? Maybe two? In any case, I am not sure what it is this morning, but I may eschew my normal level of link and news consumption to instead focus on my own projects and writing.

I'm not sure if this is related, but I had a real difficult time with the weekend's temporality. I kept thinking I had more weekend left. On Saturday I felt like it was Friday. And on Sunday I felt, until quite late in the day, that I had another day to go.

That's the morning outlook for the day, we'll see how the day goes.

Auto Embeds for Books

This afternoon I coded a book embedding function for this blog. Now I can input a book's ISBN inside my blog's custom embed tag and it queries Google to get book title, author, and book cover. With that stuff, it generates a quick embed into blog posts.

I am also linking these embeds to bookshop.org rather than Amazon, however there is no API for the site as far as I can find. So I am using the simple "send you to search their site with the ISBN" which works but isn't ideal.

Error with Newsletter Emails and Special Characters Finally Fixed

Sometimes you're just about to go to bed and one of the smaller bugs that you know exist in your code grabs you. I had already done some coding on the blog, there wasn't anything notable for you all so I wasn't going to mention it.

But the bug that grabbed me is reader-facing, specifically to the handful of folks who read the posts on the blog in their email everyday. There was an issue where some special characters were showing up wrong and giving obvious UTF encoding errors.

For example this excerpt of a headline: "Decolonize New Zealand’s Name"

Showed up as: "Decolonize New Zealand’s Name" (bolding for effect, not part of the actual bug.)

Not something which was the end of the world, and also something which I had fixed with the rest of the blog already, from input, to database, to publish, it's all UTF-8 safe and coded. It was nefarious and I couldn't figure out where it was coming from. I was befuddled but I just let it be, it wasn't an end of the world thing - but I wasn't happy about it.

Well, tonight, the blindingly obvious answer struck me out of the blue. The error had to be in the email itself, specifically in the encoding information I used when sending the email. And, sure enough, after adding a UTF-8 header and meta tag to the emails, all appears to be fixed!

With that, time to publish the fixed newsletter and head to bed.

Screencast recording failed, will try again

Tried something new this morning, I did a screen recording as I went through my RSS feeds and read the morning's news. Foolishly, I didn't check to make sure the recording setup was working since it was the first time doing it. After I finish, I open the video to discover it had recorded my mouse movements on a black screen and no audio. So... that didn't exactly work as expected. But, I might try it again another time. We'll see.

Custom Paths

Making tonight an early night after yesterday's (admittedly) productive insomnia. After work, went out to do a quick bit of errands with the wife, and after we got home and watched What We Do in the Shadows I got back into my code.

Mostly finished my taxonomy rewrite - There is still more to do, but we're back to parity with the previously (badly) written page and we've added the ability to define hierarchy in the tags. Though the backend doesn't yet do anything with that information. That's phase 2 of this bit of work.

Custom paths - Unlike Wordpress and most other blog engines, this CMS was conceived for links and Twitter-esque posts (the very first name of this blog engine was 'Blips' before I moved to 'Glowbug' as its name.) I've been expanding Glowbug's capabilities as I use it more, but due to this original design concept I didn't give posts their own pages by default.

Most of them exist only on the main index (while they're new) or on date archive pages. However, I did eventually add the functionality for standalone posts - but, again, I opted for maximum simplicity at first. Every post is given a unique hash string identifier (for example, this post's hash is 'fa340ae8'), and up to now that string has been used when generating the page urls. Simple.

Tonight I started the code to allow me to set custom paths. It's probably 60-75% of the way done. I can add them. And when I publish it sets the URLs correctly across the site. But the work, as always, remains on the admin side. Things like the paths still don't show up correctly in the admin panel, or needing to be able to edit paths once published, or when I have a post drafted but not published, that currently doesn't handle paths properly. But - we're getting there.

Delete ghost posts - As part of the above work, I uncovered an overlooked bug. The way the site published posts has, up to now, been entirely additive given that file paths were all fixed. No generated html files were being deleted. Again, this structure was perfectly find under the original design specs. But adding standalones is when this became a bug. Once a standalone post was published specifically.

Standalone posts could live on as ghost posts after I deleted them in the blog's database. Because, the way it was written, even if I deleted it from the database, it's generated html file would remain. I fixed this by adding code to remove any standalone files when a post is deleted. I still need to catch when a post path is changed after publish, and some other corner cases. But as we have a userbase of 1, I can be confident that that is not a major problem I'm going to run up against in the short term.

And that's it. I'm calling it quits for the night. Sending out the day's newsletter and heading to bed to read until I fall asleep and make up for some of last night's sleep debt.

Insomnia into Code

I suffered some pretty horrendous insomnia last night (speaking comparatively for myself.) But the good news is I've really been enjoying coding in Javascript and I spent most of that time doing it. I expect later today to be in a pretty dour mood given the lack of sleep and the heat, but I'll survive.

I spent most of last night working on basically a complete rewrite of my taxonomy system for posts. I'm adding hierarchy to tags, and also completely rewriting the way the admin page that manages them works. I'd say I'm about 75% of the way there, and then I'll dig into reorganizing them. Once they are reorganized I'll be changing how they function in the backend when I'm writing a new post (for example, when I choose a tag that has a parent tag it will auto-add that tag to the post), and then I'll be experimenting with how they're displayed on the front end.

This will also likely mean that I begin making the tag archive pages. Which has been on my to do list for a while, but I have just held off.

One of the "gardening" tasks for tags is to go through and catch typos (I had both "conspiracy" and "conspirazy"), or when I use synonyms (I decided last night to use "programming" instead of "coding" and also that when a tag is a verb, I'll use the -ing form of it.)

One feature I realize I need to add in my admin tool is that when I merge tags, rather than simply deleting the old one I should save it and what it's merging into. So, in the future, if I try to add that tag again the system will be able to autocorrect it to prevent the "weed" tag from popping back up. Not doing that now, I've added it to my backlog of things to add. Adding that feature now requires a bit more back-end reconfiguration and I don't want to rabbit hole on this all, I'm hoping to be done with my current run at the taxonomy system later today. Who knows though. Maybe another night of insomnia will make it happen.

The next big thing is figuring out the right level of complexity to enable with nesting of tags. For example, I could group all of the programming language tags under the 'programming' tag - that makes perfect sense. Is it worth grouping all the tags which are names of individuals (Boris Johnson, Joe Biden, Elon Musk, etc.) into a group under 'Person'? I don't think so, at least not as of now. Or I have a slew of "us politics", "us history", "us soccer" - do those go under "USA"? As I'm working on implementing the tag hierarchy, I'm thinking about these things and trying to decide my feelings on them.

After this bit on taxonomy, I think the next "big" feature for the blog is to add a photolog of sorts. I've never been a consistent photologger, though I've tried a few different photologs over the years. I also never really adapted Instagram as a regular place to post. But, I do post photos here from time to time, and I could add an auto-tweet for new photos, etc. It's in the backlog, we'll see when I get to it. I still have another small coding project that isn't part of Glowbug to distract myself with.

As far as my enjoying working in Javascript: What I mean is that I'm enjoying working without any library. Historically I've relied on jQuery when writing complex Javascript, but I've come to realize it isn't needed anymore. The language has come a long way over the years, especially with its growth as a backend language rather than just for browser frontends. I'm doing things with native JS which I never would have been able to do without wanting to gouge out my eyes and it's as elegant and simple as jQuery.

It's just really impressive to dive in and really appreciate how far the language has come. Well done to its maintainers.

Tweaking the Newsletter

Spent a little bit this morning tweaking the newsletter code. I cut out some empty space to make it look better.

Also, I've decided to invert the order of posts. It has been publishing posts newest to oldest for the newsletter, I've decided to invert it, so the emails will now read chronological through the day. It hasn't happened since I started the newletter, but there are some days where I do multiple entries on a developing news story. And I think for the purposes of the newsletter, this makes more sense.

I'm still trying to figure out why some UTF8 characters don't encode right for it. Not sure where it's getting mucked up.

And, sort of invisible to you, I'm changing the creation process for the newsletter. Up to now, the email sending has been a quasi-automatic process. The system would automatically generate it via a cron job. But then the newsletter tool still requires me to manually approve it. So, depending on how late I stay up it might go the same night or first then in the morning when I wake up or, honestly, it sometimes goes out when the dogs wake me up to go outside in the middle of the night.

In any case, I've now made it so that it will be a fully manual process. The intent being when I've made my last post for the night, I can click a link to initiate the process. I dislike the fact that more often than not the newsletter for yesterday is arriving at 9am Eastern for most people. There is of course the chance that this new process is simply not going to work and I'll keep forgetting to send it. I have an idea for a more elaborate system which has the safety catch of sending it each night if I haven't manually done so, but I don't know if that system is needed yet. We'll see.

Adding Commenting to Glowbug

I find myself thinking about adding comments to the blog in an effort to foster it's community growth. If I did it I would want the following:

1. Identity - Some sort of identity tie. Not trying to force real names, but something that lets me force a user to be identifiable across comments. - Lessens spam and furthers the growth of site.
2. Lightweight - ie not Javascript driven - I could turn to 3rd party tools (like Disqus), but speed is a big part of this blog. The whole reason I switched to it being that every page is its own html file and not a server built page. So loading big external JS is not interesting to me.
3. Threading - I would want a simple level of threading of comments. Maybe it caps at 2 or 3 depth, but I don't want just an endless scroll.
4. End of life - As with my Twitter, I am growing to believe that a lot of content is not useful or beneficial to remain available forever. I haven't implemented an EOL for posts on the blog, but I have been thinking about it. And in that same vein, I am thinking about it for comments.
5. Upvotes - A way for viewers to promote and support a comment they like. I don't just want a thumbs up or up arrow. Right now I'm thinking there are two options "Interesting" and "Informative." Readers could vote for one or both.
6. Troll Fighting - To be big enough to get trolls is still a ways away for the blog, but it would be prudent to plan for it now and have tools in place to avoid brigading, spamming, hate, etc. whenever it did eventually arrive.

I'm also trying to divorce myself from the standard model of comments where the comments are just at the end of a post. What if the comments were set up like page highlights and folks could comment on specific sections? What would that look like? How would that work?

This is what I'm pondering. We'll see what, if anything, comes from it.

A small editorial note for the blog

This will be overlooked by most people, but one thing I've been chafing at on the blog is how to mark when I'm reusing the link's headline or title vs. when I'm editorializing my own. I've settled on a simple (I think) structure moving forward: when I'm using the link's title or headline I will be enclosing it in double quotes. My editorialized titles will not.

Simple.

Morning Blog Coding

This morning I spent a little time tweaking the CSS on the site. It started just because I noticed a quirk with my embedded code blocks, if the line goes too long it breaks the layout. I'm still trying to figure out the fix for it. As I was working on it, I also realied I was not loving the font of the site - especially when it was used for code. So, I spent a little while tweaking it. Still not perfect, but better.

As part of this it has also become clear to me that I need to change how CSS is handled. Currently it's a template file, so updating it required publishing the entire site. That got annoying. So I finally took the first step toward section publishing, the idea that when I post an article, I don't need to rebuild the entire site.

For today, I just added the ability to define specific classes of templates I wanted to publish. So now I can specify if I want to just publish the frontpage, or the dates archives, or the individual posts, etc. Primarily this is useful for things like this morning, where I am focusing on modifying templates and want to just refresh what I'm working on - but it is a step towards the smart publishing I want.

Unshortening URLs in PHP

In an effort to expedite my posting of links from my phone, yesterday I added a function which expands shortened URLs in the blog, so I don't have to do that on my side. It took a little testing to get it to work right. I found examples online were incomplete, often written only for cases that definitely used shortened links and that did not handle when the link wasn't shortened, etc.

Sharing where I landed, in case it is useful to others.

function unshorten($url) {
    //Checks if the variable appears as a URL
    if (!filter_var($url, FILTER_VALIDATE_URL) === false) {
        //Fetch the headers of that URL, requires PHP 5+
        $headers = get_headers($url, true);
        //If the URL is being forwarded, it will have the 'location' set in the headers
        if (isset($headers['location'])) {
            //Overwrite the provided URL with the new destination
            $url = $headers['location'];
        }
    }
    return $url;
}

And voila. After less than 30 minutes of coding work, I adapted the publishing of the RSS feed into also having the blog output a JSON feed.

I suspect the JSON Feed formatting will prove especially useful when I get around to making an Android app for myself so I can more easily post and share things from my phone to the blog.

Daily Newsletter

In my continuing efforts to let more people read this blog, I've implemented (hopefully) a simple daily newsletter (sign up here. It will send in the evening and include all of a day's postings. I am purposefully keeping the design basic, aiming for ease of consumption. There will definitely be some glitches to work out as we go, but here's hoping it proves useful for folks.

I am using an old mailing list that I had set up on TinyLetter. I've removed almost everyone who had previously subscribed since this is an entirely new effort. And, given that it's going to be going from one email in the past three years to a now daily cadence, I felt it was prudent.

Today has been spent doing chores in preparation for our BBQ tomorrow. In addition to those though, I've also done a repair on my La-Z-Boy recliner, something I plan to write up or record a video about for other La-Z-Boy owners.

And in between those things, I've also done a little bit of coding. Most of it was behind the scenes stuff, such as fixing my image uploader (it was supposed to be auto rotating and resizing images, but hasn't been) as well as coding in for the blog to try to include favicons of domains when linking to another site (only from the homepage, for now.)

It works 100% for sites which follow the standard structure and formatting, but it isn't yet smart enough to catch when sites take shortcuts or don't exactly follow the prescribed formatting. I'll keep fiddling with it, as always.