A new writing project has started. I have no idea where it will go or what it will turn into. But I have been relistening to Dungeon Crawler Carl and it has made me want to find a story that is about someone who is thrust into the role of space captain. It doesn't have to be litrpg but that seemed the most likely framing for it. And to be clear, I mean someone who isn't familiar with space travel at all.
After looking for a little while, I didn't really find a story that filled this niche. Which means... my brain started creating one. So I've started writing it. I don't envision it as a full novel, maybe a novella, or maybe a series of shorts. We'll see. Who knows. It'll depend on my drive to write it before I get distracted or lose steam.
|
Podcast Middleman Dev
Hammered the majority of the podcast middleman out this morning thanks to Python. A few things still to fix (like the fact The Daily has a 12.5 meg RSS file and I am not looking to archive their entire backlog) but the core functionality is there and it's just about trimming off what I don't locally archive, etc.
|
Working on 'Wyrmling'
Spent several hours so far this weekend getting my custom collection management project. My goal is for it to be an open source self-hosted solution others can use. It's meant to be flexible to use for any set of collections. For example, Katie and I will be able to use it for my Magic cards, her PEZ, my D&D books, our vinyl albums, etc.
You may recall I shared a database diagram a few years ago. The database structure has remained largely the same with some changes since then. The core concept remains the same: there are collections and there are libraries. Collections are what you personally possess. Libraries are the type of collection and its things.
For example, a library would be 'Magic: The Gathering' cards. Pulling that data in from an outside source to include every Magic card. The collection would track which cards I personally own. You don't have to have/use a library for your collection, you can just enter your own title and description, but the library makes things simpler.
A lot of the work has been fleshing out the collection and library database tables. Things like adding fields and foreign key relations, etc. I just added a table for tracking changes to collections, so you can see adding things, removing things, etc.
I'm getting close to being ready to begin building the UI in earnest. That's going to be the brunt of this work.
|
Sometimes your blog breaks for no reason
I don't know why.
A few weeks ago, I rewrote how the blog generates what I call the "link tails" which is when a link is followed by the (domain.com) information. It was originally part of the publication of the blog and thus embedded in the html files directly. But then I decided I wanted to move it to be javascript.
Honestly... I'm not sure why. I had a reason. I'm sure of it.
In any case, it wasn't working today. I'm not sure what happened, but it appears to be an issue with how I was filtering the links. Not every link on the blog gets a tail. Only ones which go off the blog (so no trickjarrett.com tails) as well as no links in the sidebar.
And something stopped working with that such that no links were getting tails. I did the filtering a different way and it's all fixed.
Also it looks like there was a reversion of the CSS file. I'm not sure why, but something got lost and I had to recreate a bit of CSS. My best guess is that I accidentally overwrote the site's CSS file with a local copy which was not the most recent, as I sometimes tweak the CSS file on the server. Oops.
In any case, it's fixed now.
Edit: Regarding Wikindle project
It finished running overnight. I spent a little while coding a cleanup function which scans downloaded files and removes ones I don't want. This is one of the features I mentioned wanting. So I've got that figured out. And it blacklists articles so I don't re-download them in the future.
I started working on having it generate cross linking between articles, but there were enough bugs that I stopped and decided to come back to it.
For now, I consider the project done and stable. There are improvements, but it's time for me to move back to working on the blog and on Behemoth.
|
Update on last night's 'Wikindle' code
Code ran without issue, though the formatting was slightly off.
This morning I hammered out the code to grab the top 100 most popular entries from yesterday and add them to the archive. I'm not sure how useful that will actually be, a lot of those entries are pop culture (am I really going to need an entry about the new Kraven Marvel series?) But we'll see. It isn't like this is a major space hog.
Last night snagged roughly 8,000 entries and it took up 250 megs. Plenty of space.
One thing which is lost in this process is any cross linking. I'd love to go through and add that back in, or even better figure out how to best avoid that bring stripped out from the start. We'll see. In any case, a fun diversion to distract me.
|
Wikipedia on my Shelf Progress
At this moment my PC is downloading nearly 10,000 entries from Wikipedia as part of my idea of a locally hosted version of the encyclopedia. I'm making use of the Vital articles project, specifically level 4, which is roughly ten thousand entries on various topics.
I cobbled together a python script to pull from the API, parse the HTML to markdown, strip any lingering tags (such as spans, abbrs, etc.)
It isn't perfect, no images from the entries are brought over. I'll work on that further in a future iteration.
Not sure what to call this project. I called the folder "Wikindle" as a smerging of "wikipedia" and "kindle" but I don't love that name. I'll play around with it.
I also have an idea for this to be a "living" archive. Where perhaps it is a cron script which runs nightly for that day's top X entries, and snags them, accruing more and more notable content over time. Obviously quality will vary, but we'll see how it goes. Then, every few months, I update the Kindle.
Lastly, my observation as I work on code this evening. It remains comforting that ChatGPT struggles with some very basic coding concepts. I know lots of people worry LLMs will lead to the end of programmers but that simply isn't true as far as I can see. At least, not without more massive steps forward.
|
Behemoth has begun
Last night, I finally truly began work on what I've come to call 'Behemoth.' It is the generalized collection management app that I've been thinking about for a few years now. Originally I was going to call it CeMeNT for "Collection ManagemeNt Tracker" but the more I thought about this project and how big it would be, I decided to give it a more impressive sounding name.
The idea centers around a flexible abstract database design which would allow a single centralized system to manage collections of all different types. The two most notable for Katie and me are her PEZ and my Magic: The Gathering cards. (Disclosure: I work for the makers of Magic, Wizards of the Coast.) But we could also track our vinyls, books, boardgames, and metal lunch boxes in a system like this.
Elwood woke me up at 3:30 this morning and I sat down to re-diagram the database schema. I came up with initial schema design back in December of 2021:
And at the time, when I first shared it, I framed it as a concept that I wouldn't write because I knew it would be a very large project. So I sat on it for the past 18 months, occasionally it would come back to mind and I'd think more about it. Over that time I'd refined the concept a bit mentally, but I had never sat back down to redo the schema.
Well, with this morning's wakeup, I found my brain iterating on the Behemoth project and thinking about the database.
So, I sat down and rebuilt the schema which is a bit more robust version 2:
The biggest change from V1 to V2 is that I formalized a design where you have the 'library' which is the "master list" of possible things in a collection. This simplifies a few things in the overall management, though it introduces its own issues.
A lot of the "version" details are tracked in the library rather than the individual collection. This makes sense as you'll have new items which refer to the library. But, collectors love to collect random things. What if your collection has something which isn't part of the larger library, whether by obscurity, or maybe something like an artist signature - how is that tracked? I'm not sure yet. I'm letting my subconscious chew it over right now.
Another thing is that collections come in two varieties. I have come to think of them as the vertical and the horizontal. Verticals are the ones which are "easy." They have a very clear backbone, whether it is "Magic: The Gathering cards" or "PEZ dispensers." Horizontal collections might be for an IP or fandom, where it is much more broad in varieties and details. Think of a "Star Trek" collection which might include DVDs, books, clothes, toys, etc. Currently Behemoth is well suited for vertical collections, and so I'm trying to figure out how best to handle horizontal collections.
To that end, I'm still trying to come up with different collections, and find corner cases which might be out there that this structure can't handle.
Even as I seek more unknowns, there are a few known things I need to add and account for. That said, I am quite happy with the concept V2 currently represents.
Two of the things I know I still need to account for:
- Commenting / discussion - Allowing additional conversations on the collection.
- Transaction tracking - Tracking when we buy or trade for things in the collection. Tracking the acquisition price, where we got it, etc.
While the database comes into focus and crystallizes, I'm also preparing to actually start coding.
I'm going to build this in PHP using the Laravel framework. I'm still deciding on the CSS, probably Tailwind. The database will be MySQL. I debated if I should use MongoDB to allow it to be more flexible, but the truth is I don't know MongoDB well enough to schema it properly. So I'm sticking with what I know, MySQL.
We'll see how this goes.
My intent is to open source the project and allow others to contribute to it. In theory. If it works and I'm not embarrassed at the quality of my code.
|
Testing the Corsi-Rosenthal Box Fan Air Filter
Katie and I tried one of these a few years ago and were underwhelmed. I think it was our box fan design as I felt like it wasn't really moving a lot of air through the filter. And I recall seeing someone online note that these fans had a propensity for drawing air back through the corners outside the fan blades. So we abandoned it and got a small unit from Ikea.
Notable from this video, they are all blowing in directions other than the floor, or are lifted off the ground. Mine sat on the ground and just had four filters. So, it sounds like I might have just built ours incorrectly.
Oops.
|
Side project on my to do list: A website where you enter a word or short phrase and it returns every translation for it possible.
Sometimes I just want to see a word or phrase around the world and not have a specific country or language in mind.
|
Writing Update
I had taken a few days off writing, letting the story percolate in my mind. I had reached a point in the story and couldn't figure out how to proceed forward. I was stuck and needed to figure out what to do. And sometimes, the right answer is not to plow forward. I eventually decided the right course of action was to get delete some stuff I had written and I'm glad I did.
Once I removed the obstacle, I jammed out roughly 3,000 words this evening. The log sheet will only show 2,700ish, but that is because I had deleted the problematic section two nights ago but didn't log it. A session with a negative word count would wreak havoc on the formulas for my tracking sheet. So, instead, I have a slight undertruth in the number for tonight. A fair trade off for avoiding the headache of making it so the sheet can handle the rare negative wordcount session.
Additionally, I'm not killing myself writing this. This isn't NaNoWriMo. I've set myself a very achievable goal of 2,000 words a week on this book. This gives me license to only write a few times a week. Sure, I can do more if I find the time or motivation, but I'm not going to be wracked with guilt if I don't.
|
How to make a chess engine
Saving this for a future project. I've had an idea that there is a teaching tool waiting to be made for chess. Have an engine evaluate positions and identify squares of frequent conflict in the lines it examines, as well as pieces most likely to be traded, etc.
I have no idea if that would actually be useful. But it seems to me, a training tool which had players learn to identify these things, might be good? No idea honestly.
|
A New RSS Reader For Me
I have, for over a year now, scanned and read tens of thousands of items from RSS feeds through my FreshRSS installation. As of this writing, I am subscribed to 114 different feeds which I break into eight different categories. Though, the truth is, I don't use the category organization nearly at all. And part of that is because, it isn't relevant to my consuming the feed, the categories are there for me to manage my feed.
I will check my reader several times throughout the day, usually seeing between 50 and 200 posts when I do. And I don't bother going site by site or anything. I just use hotkeys and quickly jump to each entry, scanning through them for interesting stuff. I'd wager 80% of the links I post here originate in my RSS feed now. I love it.
That said, I've had a germinating idea for a new RSS reader-esque site/tool sitting in my head for weeks now. I had a few different pieces and ideas, and today's blog post by Dave Winer stirred it back up for me.
Within an hour, I had a list of features & ideas jotted down. The original idea was something more akin to a Tweetdeck style multi-column layout social bookmarking site.
But Dave's post caused me to reframe the idea and repurpose the UI. I do think this idea is more than just an RSS feed reader. I think of it more as a personal information hub. I'd integrate other API services, such as social platforms (pulling tweets in?), sports, weather updates, etc. Maybe even email.
The screen layout is in columns, though I don't yet have sizing or design concepts really in mind. I think there are two main columns and the ability to add more.
The main (to me, that means left, but we'll see how it ends up) column of this is the "Fast Lane" (working title) - which would be the most recent unread feed of stuff sorted chronologically. It's the firehouse of information.
I think the next column becomes a "Queue" list. You can flag content in the fast lane, and it gets pulled to the right so you have more time to peruse it. But the stuff in that column has an expiration. I've toyed with this idea when I was using Glowbug's draft post functionality as a quasi "to read" list. I didn't want to get to overwhelmed with things, so I was instituting a 48 hour timeout for this column, after that time it got dropped, assuming that since the item was not worth the effort to read in that time, that it was no longer important. If I really wanted to read something, I'd either read it immediately, or save it more permanently to come back to. I think that same idea would go into effect on the Queue. Though you'd customize the timeout time.
As I said, you would also have the ability to make different lanes. Maybe you want to filter on keywords, or a lane just filled with specific data sources. Or if it does integrate email, then you have a dedicated email inbox. That should be doable.
Now, also as above, sometimes I will just want to save things for a longer term purpose. Maybe it's a future blog post or maybe it's another project. That's when things get parked. And Parking is the more full "let's archive it, tag it, categorize it, etc." Save snapshots of the page, etc.
That's the core of the idea.
All of that could be just for an RSS reader style implementation. But I think this idea could also be more of a central hub. As someone who hungrily consumes information and data, this structure would really appeal to me. And it would allow me to combine my RSS reader and my Wallabag/Pocket content. You could come across a page or article outside of it, and then immediately queue or park it. The focused reading would need to be aces, optimized for ease and comfort of reading across devices.
And who knows what else there would be. Lots of integration ideas to play with and this is already quickly becoming a behemoth project. But... I'm not intimidated. I'm excited. God help me.
|
I have had an idea for a new RSS reader. It's a tool I want, but I don't really want to build. Currently my RSS reader pulls in 97 different feeds from varying sources, mainstream news, regional news, blogs, link aggregators, etc. One of my frustrations is when I see multiple articles on the same topic sporadically through my feed. Granted, this isn't a huge issue, but each successive article is an increasing volume of noise in the feed.
It makes sense, because I am purposefully pulling in diverse feeds, multiple places are covering a topic. And if it is an important topic, then it is good for multiple perspectives, etc. This issue is one thing preventing me from actually pulling in more feeds, with this system, the posts on the same topic suddenly become a potential value add rather than a detractor.
But at the same time, I'd really like to smooth the feed and instead see a singular entry on the topic with a prominent inclusion of "these other posts are discussing it as well."
So, I've had this idea for a while. Last night I think I found an open source project which would get me much closer and be, at the least, a first tool in looking at posts and identifying ones which have overlap. I will have to explore it further this weekend.
|
Last night I pushed the second "big" feature for my Pick'em site. It went shockingly smoothly. I expected it to be mostly good but pretty much the only hiccup was an errant curly brace. I was shocked.
The code is the about 70% of the full feature and is mainly the functionality that is needed for the league to continue to function as planned. This weekend I'll work on the remaining 30%, which is largely administrative tools and integrating it further into a few other nooks and crannies on the site.
With the deployment though, I find myself deciding on my next coding project. I think I'll do a bit of work on 'Clerk' which is, right now, just my weight and health tracker. It needs a bit of love and I want to expand it's functionality some. But that isn't too large a project. I think after that I'm going to start work on what I'm calling 'Kontakto' which will be a personal CRM tool. It will be my second foray into making an Android app.
|
Project Statuses
Taking a cue from Brandon Sanderson I thought I should start doing reports on the status of my projects. He has status bars which just get updated on his site, I'm not doing that (yet?) - I'm doing a post which summarizes the status of various projects underway. My intent is for this to be a monthly check-in.
Coding Projects
Pick'em
Status: Active
Summary: The site is up. There are two major features which I need to work on, one of which is underway. They are: Redoing the Match page & Implementing split-season scoring. The former involves rebuilding the page where users view and pick matches, as well as implementing the pick logic client side so they don't have to try and wait for the server to reject their picks.
Next Steps & Timeline: Finish my work on rebuilding the match page (2 weeks?) and then work on revamping player scoring to implement split-season scoring (finish by end of April)
Glowbug
Status: Active, mostly idle
Summary: My homebrew blogging engine. The work this week was the most I've done in a long time. The system works, it's just about making it better at this point.
Next Steps & Timeline: Nothing major, just coding when it strikes me. No timelines on it.
Clerk
Status: Mostly idle
Summary: Currently my personal weight and exercise tracker, but long term it's meant to be my full personal hub for managing daily life. It's up and I use it daily (need to weigh in today, in fact.) There are some active bugs which need squashing, but they are low priority.
Next Steps & Timeline: Bugs will be squashed when I need a coding distraction from whatever I'm working on.
Fixture Picker
Status: Needs rewrite
Summary: My fixture picker works but it isn't great and it needs a rewrite. This project lost a lot of stream as I thought it might have business viability but some surveys and research led me to abandon it as not overly viable.
Next Steps & Timeline: Unclear. I might abandon this project altogether simply to clear the deck for other things.
Cement
Status: Idea phase
Summary: A homebrew collection management tool. This promises to be a coding project near the size of Pick'em if not larger. I have done a lot of thinking and have a database schema but I haven't begun on this project. I'm earmarking it as a project for the 2nd half of 2022.
Next Steps & Timeline: Want to start work before 2023 and have this be my primary project next year.
Secret Project
Status: Prototyping
Summary: There's a fairly new project I've started toying with in the past few weeks. As I think it might have commercial viability, I'm keeping wraps on it and have only discussed it with a few people.
Next Steps & Timeline: Unclear, doing research and planning.
Writing Projects
Urban Fantasy 1, "Thorn" - My NaNoWriMo book for 2020. It's "complete" and just needs editing and tightening of the plot. Thorn is the working title I started with, I'm not settled on it as the final title.
Urban Fantasy 2, "Untitled" - I started it for NaNoWriMo 2021 but have not finished it. I hit a bit of a wall and am struggling to find the next section of the book.
Sci-Fi 1, "The Stars in the Canopy" - I've had the idea for a science fiction story but have not really started on it yet.
|
Coding Updates
This will be a bit of a meandering post, not in the sense that I don't know where I'm going with it, but that I plan to hit a few tangentially related topics along the way. Obviously my last post was about how I had revamped my project boards for my main coding projects, so coding is on my mind.
I have found Minecraft scratches a lot of the itch I serve by coding on my own projects. Coding projects, to me, are my equivalent of the people who sew, or do woodworking, or work on cars, etc. It's a hobby and craft that I find satisfaction in the work and the end product. And Minecraft's world building fills much the same mind space for me. It's gotten to the point where I have to force myself to put Minecraft aside and turn back to these side projects lest they fall by the wayside and into disrepair.
It's also notable that the only one of my code projects which is used by other people is the main motivator for this. I run a self-hosted custom MLS Pick'em league for friends and acquaintances. With the next MLS season on the horizon, I have to get moving on updates and fixes in preparation for the next season.
And so, after several months of going hardcore on Minecraft, I have to move it to a back seat for a bit and instead return to my hobby of coding in the hours after work.
My current plotted course for this work is:
- MLS Pick'em - It will be my exclusive (well, probably) coding project until the new season kicks off. Then I'll let it fall back in priority to maintenance rather than working for new features. Putting arbitrary dates on it, I'm expecting to focus on it until middle of March and then it gets backburnered.
- Clerk - My personal tracker, it's a small project and I have a few updates and new features I'd like to add. This stuff should honestly take a week or two, so probably through the rest of March.
- Glowbug - This blog hasn't had any serious work from me for nearly a year now. And I do have work I want to do on it. This ranges from front facing layout and design, adding visitor search, importing tweets into daily archives so they get archived here, and other sundry items. Assuming the time estimates for Pick'em and Clerk are correct (and we know how good coders are at estimating work time) I think the work I want to do here would probably be all of April and could extend into May depending on how ambitious I want to be.
- CEMENT (CollEction ManagEr aNd Tracker) - I wrote about a flexible collection manager tool last year, and it is still an idea I have been turning over now that my Magic collection is sorted. So, if I'm still actively thinking about it, then it is time for me to probably act on it. It would have utility for my Magic collection, Katie's Pez collection, our boardgames, and various other collections we have. In scale, I think it would end up being closer to Pick'em as far as size, there's a lot of front end which would need to be done for it. We'll see. For Cement, I can easily see it being a 3 month project to get it up and running and then it becomes about maintenance and adding new features.
Assuming my time estimates are correct, and that life cooperates (meaning there is nearly a zero percent chance I'm right) I have coding plans up through July. We'll see.
|
New Project Idea - Scrib
I make frequent use of Google's Keep as my place for short easy notes. I also sometimes use Samsung Notes, as a by product of my writing on my phone (Note 9), but those are almost entirely scribbles as I lay in bed and want to write an idea down.
I want a note taking tool which lets me grab random thoughts and save them. They might be quotes, or ideas, or reminders. They might often be half-baked.
This idea actually came as I am rethinking my personal organization and figuring out how best to build it.
Goals
- Be able to scribble a note within two clicks on my phone.
- Make the referencing of these notes more useful
#2 is where both Keep and Samsung's notes tool falls down. They are just the digital equivalent of yellow sticky notes. I want to have it be smarter at organizing notes by date, as well as by identifying keywords. If the word 'D&D' appears in multiple notes, then I want that word linked to a page which pulls all notes which use that word. I'd like to do it even for phrases, but am not sure how taxing that is processing wise.
I'm going to keep doodling on a design for the project, and then I'll add it to my coding roadmap.
|
