Marcus Obst

Moving from Tumblr to Self-Hosted Wordpress Pt. II

Mon, 09 Dec 2024 00:00:00 +0000

Moving from Tumblr to Self-Hosted Wordpress Pt. II

Since the move from Tumblr and my new blogging aspirations meant to deal more with WordPress, I have a few more things to share.

My goal was

Serve public and private posts in the same stream
Hide Private posts in the listing, but if you hit the private post directly (via rss, notification), show a message to register/login.
New registrations have to be approved
Spam Registration/Login Protection
Subscribers should be notified by mail when a new post happened
Free extensions

1. Membership Extensions

I first thought I should use Wordpress Jetpack Subscriber because it came with the whole "content protection, registration, and login as well as notification" jazz. But I figured, for a semi private blog I didn't want to ditch tumblr just to be depended on Wordpress servers (same thing, I know).

What speaks for the Jetpack solution, it just works out of the box and looks good. No unstyled forms and elements, no short codes, everything is wired up correctly.

Other membership plugins offer similar things, but often stick out like a sore thumb if you don't get the CSS right. Or the templating is weird.

Anyway, I ditched Jetpack because the subscribers have to go through Wordpress.com, a hard to sell and ahgain, Wordpress is going to change their service if they want and suddenly it's paid only.

For a work project, I used Ultimate Member before, but in order to learn something new, I tried out Simple Membership.

Both offer protected content, login and registration, user management and some paid content stuff (usually in their PRO offering).

Simple Membership did what I wanted in the free version, but I ended up debugging their code, just to find out the registration form can't be overwritten in a child-theme. Odd inconsistency, but it looked like they were improving this old code.

Anyway I wasn't happy and went back to Ultimate Member, since their forms look decent by default and are easier to modify (form builder is included).

The whole admin UI is also a bit nicer and easier to follow.

How to protect several posts without going into each post?

The trick is to create a WordPress category “Private” for example and protect this category. This is the way it works in both plugins. This way, I was able to add posts to the “private” category in bulk!

2. Notifications about new posts

Notification Master is offering that. It works fine for email.

3. Filter out protected posts from listing

Advanced Query Loop extends the WP Query Loop component and offers more ways to filter content.

Conclusion

I have to say, I was a WP hater in the past, but despite some annoying things (WP drama 2024, the constant upselling of extensions etc.) I really like it to quickly set up a complex webapp/website that you can't come up with, with Kirby or Statamic or Craft CMS (who like to take some of WP's share). If the out-of-the-box stuff is good enough, go for it. Damn, even the native template builder is pretty good, once I understood it. So, not the old PHP blog anymore, yet all there is needed is a simple shared PHP 8.x.

Moving from Tumblr to Self-Hosted Wordpress

Fri, 06 Dec 2024 00:00:00 +0000

Moving from Tumblr to Self-Hosted Wordpress

I'm a former Wordpress hater, but it's undeniable how fast one can setup a working website. And especially a blog (even if nobody uses that anymore).

First, I love Tumblr. I love the tech. It's still the easiest and most flexible network to post. Photo-galleries, media, text. It just works, and it's not a walled garden, but a public blog/website if you want to.

What I don't care about is the community and the ads are annoying, but more so the fact that I don't want my private travel blog being on there. My travel blog that only two people cared read, maybe just one that managed to bookmark the site.

Anyway, here is what I learned about how to importing a tumblr blog worth of 12 years of traveling the US and Europe.

1. Wordpress importer

If you google for a tumblr to Wordpress importer you'll get a ton of results talking about this plugin: https://wordpress.org/plugins/wordpress-importer/

It's tried and tested and does what it claims to do. Transfers your Tumblr Blog to Wordpress.

Where it gets tricky is the following:

2. (no title) Post Titles

Tumblr didn't use post titles as a specific field. It's just a formatting, so the list of posts in WP backend looks non-descriptive with all the (no title) posts.

The solution to that was a script in functions.php that writes the title.

/**
 * Automatically set empty post titles using first 13 words of content
 */
function auto_set_empty_titles($post_ID) {
    // Get the post
    $post = get_post($post_ID);

    // Only proceed if the title is empty and we have content
    if (empty(trim($post->post_title)) && !empty($post->post_content)) {
        // Strip any HTML tags and shortcodes
        $content = wp_strip_all_tags($post->post_content);
        $content = strip_shortcodes($content);

        // Get first 13 words
        $words = explode(' ', $content);
        $first_13_words = array_slice($words, 0, 13);
        $new_title = implode(' ', $first_13_words);

        // Add ellipsis if content was truncated
        if (count($words) > 13) {
            $new_title .= '...';
        }

        // Update the post title
        wp_update_post(array(
            'ID' => $post_ID,
            'post_title' => $new_title
        ));
    }
}

// Hook the function to run when posts are saved
//add_action('save_post', 'auto_set_empty_titles', 10, 1);

// Optional: Run this once to update all existing posts with empty titles
function update_all_empty_titles() {
    $args = array(
        'post_type' => 'post',
        'posts_per_page' => -1,
        'post_status' => 'any',
        'title' => ''
    );

    $posts = get_posts($args);

    foreach ($posts as $post) {
        auto_set_empty_titles($post->ID);
    }
}

//update_all_empty_titles();

Uncomment update_all_empty_titles(); and then reload your page. It took a while and the script pooped (timed) out, but another reload just took care of the rest of the empty title fields.

The script was written by AI, don't @ me if you set your wordpress on fire!

Btw. Chris Coyier made a case for not using post titles.

3. Galleries

Over the 12 years, tumblr changed a lot and so did the tech behind. I think from 2019 or so on, they fully embrace the block editor. Not like Gutenberg Blocks, but some sort of WYSIWYG. Before you could write markdown, html or some WYSIWYG html with special tags. It was bad.

Then they invented some weird JSON markup called the Tumblr Neue Post Format - no clue why so complicated, but for some time it wasn't possible to edit a post in the browser, made with the app and vice versa.

What I want to say is, the imported Markup is weird. Earlier post just write in a plain WP [gallery] short code.

Later galleries are imported as HTML (which is better).

To deal with the [gallery] short code I found a script here Filter classic WordPress gallery shortcode attributes. This makes the gallery a little bit more presentable. It still looks ugly, but at least it's not just the cropped thumbnails.

Other gallery pics are just loaded fully one after another. You can go in, convert the classic editor field to blocks, delete a few blocks and convert the images to a WP or Jetpack gallery. But I would do that only to posts that I edit.

/* filter gallery output*/
add_filter( 'shortcode_atts_gallery', 'my_shortcode_atts_gallery', 10, 4 );
function my_shortcode_atts_gallery( $out, $pairs, $atts, $shortcode ) {
    if ( ! isset( $atts['columns'] )  ) {
        $out['columns'] = 1;
    }
    if ( ! isset( $atts['size'] )  ) {
        $out['size'] = 'full';
    }
    if ( ! isset( $atts['link'] ) ) {
        $out['link'] = 'none';
    }   
    return $out;
}

Alternatively, here's a script to convert plain [gallery] shortcodes to tiled Jetpack galleries.

Check the box under Settings > Media > Tiled Galleries and if they appear too small (500px by default) set the content width of the template. See https://jetpack.com/support/jetpack-blocks/tiled-galleries/

// twentytwentyfive template uses a content width of 645px
if ( ! isset( $content_width ) ) {
    $content_width = 645;
}

/* filter gallery output*/
add_filter( 'shortcode_atts_gallery', 'my_shortcode_atts_gallery', 10, 4 );
function my_shortcode_atts_gallery( $out, $pairs, $atts, $shortcode ) {
    if ( ! isset( $atts['columns'] )  ) {
        $out['columns'] = 3;
    }
    if ( ! isset( $atts['size'] )  ) {
        $out['size'] = 'full';
    }
    if ( ! isset( $atts['link'] ) ) {
        $out['link'] = 'file';
    }

    if ( ! isset( $atts['type'] ) ) {
        $out['type'] = 'rectangular';
    }

    return $out;
}

Another option to deal with plain [gallery] tags would be to do a YOLO style search & replace directly in the database.

UPDATE wp_posts SET post_content = REPLACE(post_content, '[gallery]', '[gallery link="file" size="large" type="rectangular"]') where post_content LIKE '%[gallery]%' and post_status = 'publish'

Don't @ me if you messed up something.

Also, this way you can't convert the gallery in the Classic block to a Gutenberg gallery block.

4. Hot linking images to Tumblr

Some images were uploaded to my WordPress, others (newer ones?) were hot-linked to the tumblr cdn.

To download all the externally linked images I tried out two plugins:

Download External Images In Posts – WordPress plugin | WordPress.org this one downloads all images in a subfolder of the media library and you can't see them there.

Auto Upload Images – WordPress plugin | WordPress.org and this one does the same, just that images are supposed to be available in the media library. I'm not sure if it worked, since the plugin wasn't tested on newer WordPress versions.

Conclusion

It's not as straightforward as the plugin is trying to sell itself. But's it's possible to transfer the Tumblr blog to a self-hosted WordPress instance.

Main reason was to have some sort of subscriber(my mom)-based travel blog, so my data is not public to anyone. That level of control war not possible in Tumblr.

Moving from Flickr to Self-Hosted Piwigo

Sat, 12 Oct 2024 00:00:00 +0000

Moving from Flickr to Self-Hosted Piwigo

See my personal motivation for moving away from Flickr to self-hosted Piwigo at the end of the page. Here are a few points to outline what to expect from Piwigo:

It's album-based; there's no real loose photo stream as we know it from Flickr or Google Photos.
It's mature software that's battle tested but rough around the edges with a lot of baggage (coding style, jQuery).
There is a neat iOS and Android client
It needs some adjustments, and you better know a thing or two about how to fix things yourself.

Install Piwigo

That should be straightforward. It's old-timey FTP deployment, either by one php script or by uploading the package. It's described on their get-started page.

Setup

Once the thing is running, and you uploaded a few pictures to check it out you might think, “How can I get my photos from Flickr into Piwigo, and how do I make it look at least a bit prettier?”

The answer is a bunch of plugins!

1. Activate the LocalFiles Editor Plugin.

This provides access to a few files you might want to change, mainly the config file.

Here I set the following value to suppress PHP notices or warnings in case a plugin doesn't like the latest PHP version (8.2 in my case)

<?php
$conf['show_php_errors'] = E_ALL & ~E_NOTICE & ~E_DEPRECATED & ~E_WARNING;

Other config settings can be discovered in the default config file.

2. Install the flickr2piwigo plugin.

Activate it
put in your Flickr API key and API secret
start importing either album by album manually or all images (albums will be created)

For this plugin to work, I needed the PHP error suppression because one lib in the plugin is not compatible with PHP 8.2 (see the GitHub issue)

The import was relatively straightforward.

Now you should have a working self-hosted photo gallery that you can browse in the frontend. And there you see: it looks dated af and what are these URLs /index.php?/category/album-42?

3. Search engine Friendly URLs

Add the following to your config file as described before:

<?php

$conf['show_php_errors'] = E_ALL & ~E_NOTICE & ~E_DEPRECATED & ~E_WARNING;
$conf['question_mark_in_urls'] = false;
$conf['php_extension_in_urls'] = false;
$conf['category_url_style'] = 'id-name';
$conf['picture_url_style'] = 'file';

?>

And also place a .htaccess file into the root and add this:

AcceptPathInfo On
Options +MultiViews

Make sure you have no index.html or index.htm in your root, only index.php!

Check to see if your public gallery works!

4. Change the theme to Bootstrap Darkroom.

Bootstrap Darkroom can be installed and setup from the backend. It's the most modern looking theme. Here are some screenshots:

1. Install it via the “Add a new theme” tab: 2. Activate and set it as the default, then configure the theme.

3. Select a color theme you like.

4. Play with the settings; here are mine as a reference:

That should make the page look like this:

For the secondary zoom into pictures via PhotoSwipe to work, set your image sizes as follows:

Go to Photo sizes and “show details”

Change the "XXL - huge" to something larger than the default value.

Then you can do the following:

Set full-screen (lightbox mode or whatever you want to call it).

Click on the image or the magnifying glass

And the image scales up to the XXL-huge size, as you know it from Flickr:

5. Justified image gallery.

That's what they call the “bricklayer” effect, you know from Flickr or Google Photos. It looks much better than the even cards of images in the Bootstrap Darkroom theme.

For that, you need another plugin called gdThumb, easy to install via the backend.

Play with the settings, here are mine:

to make it look like this:

Conclusion

These were the basics to make it work for me, or better yet, to make it look more modern. Most themes are from 2015 or older. It works, but it doesn't look so good.

For the rest of the setup, you are on your own.

Why did I leave Flickr and moved to Piwigo?

I've been on Flickr since 2007, when it was a cool photo-sharing community. Then it got sold to Yahoo, which offered unlimited storage, which turned it into a low-quality image dump, at least partially. Yahoo sold it to Verizon who didn't care too much about it (along with tumblr)? Anyway, it got saved by SmugMug, who put an end to the 1 TB of storage, and that forced me to buy a subscription. 35 USD for a year was okay. From then on, they raised the price and did only little things to keep the platform running and evolving. The Wikipedia article about Flickr explains it all.

Personally, I didn't expect much. I documented places (outsider art environments and things like that), and the free exposure was a nice add-on, but primarily I liked to work with the API and embedded my pictures on my website this way. At some point, I set all my images to a Creative Commons license for people to use (I ended up on Atlas Obscura and in some Kentucky tourism magazine)

This year, they raised the price again. Just 1 EUR, but I looked at it, looked at alternatives, and pulled the plug. No Flickr PRO for me.

What they offer is mainly unlimited hosting of files — I already pay for web hosting, and now for Google Photos — so I can't justify Flickr as a photo backup. I don't need any of the perks they offer (some small discounts on photo books that I can't order anyway). I also don't want to subscribe to Adobe. The community is dead or not interested in my stuff, so I don't miss anything in that regard. SmugMug made a big deal out of improving the statistics, but I couldn't care less whether my photo has been seen 33 times or 44 times. One of the arguments for using Flickr was that it doesn't use your data like Google Photos. Ok, true. And? Nobody cares about this anymore. We accept all cookies everywhere because maintaining privacy is a chore, nobody wants to deal with it. They use my photo of a dog to show my dog food ads? I don't care. I'm more annoyed by Cookie-Banner than the ads I'm blocking anyway.

Again, I liked their API. Flickr was very developer-friendly in the beginning, but all the apps people built 15 years ago are dead, and nothing new happens in that space.

Piwigo was always on my radar, and since my web host offered a one-click install, I tried it. First, I was unsure because, even though the Piwigo websites look pretty nice now, the software is dated and reminds me of some old Joomla backend. But I dug in and managed to import my Flickr photos without issues, polished the instance with some of their plugins, and even contributed a bug fix. It looks like the community has resisted complete rewrites in some framework that needs updates and maintenance all the time. Instead, you have old-school SQL queries instead of an ORM and no compose.json.

And since my engagement on Flickr was low, I don't expect more on my personal Piwigo instance. So I save money, had a little work to figure it all out and the Piwigo API is alright, I can work with that as well.

Another contender was Lychee, also a self-hosted photo gallery based on PHP. I tried it, it's pretty smooth and nice, but it couldn't convince me completely, mainly because it wasn't easy to get my Flickr photos imported and add new images in the future (I think they turned off the ability to read whole directories with images). It also is based on Laravel. While Laravel is fantastic software, as far as I know, I'm not a fan of Laravel. So it was easier to dismiss, beside the missing requirements.

Update 2024-12-06: I sold a license for a photo that allowed be to buy another 2 years of flickr pro, but not without letting the flickr support know that I'm leaving, which resulted in a 30% discount. So ~50 EUR a year, I'm ok with. Also it turns down, selecting 1000 photos to stay on (for the free plan) is as hard as selecting which photos to upload. That's a dilemma.

But since I'm local first, I can distribute photos to flickr and piwigo in almost the same step. So I guess I try

AtroPIM v1.9.21 API breaking changes for related assets

Sat, 19 Aug 2023 00:00:00 +0000

AtroPIM v1.9.21 API breaking changes for related assets

Some new breaking API changes in my favorite Open-Source PIM AtroPIM, that weren't documented (unless I overlooked it).

[TOC]

According to the changelog (with a little more context here) they changed something related to how product assets are requested.

To get all assets related to a product, I called /Product/:entityId/assets in the past, which resulted in a collection like this:

{
    "total": 6,
    "list": [
        {
            "id": "63a41a9be351041b6",
            "name": "Einbauhinweis_1380_1390.png",
            "deleted": false,
            "icon": null,
            "url": "/upload/files/7k4dp/Einbauhinweis_1380_1390.png",
            "height": 1470,
            "width": 1500,
            "colorSpace": "SRGB",
            "colorDepth": "8",
            "orientation": "Landscape",
            "isActive": true,
            "tags": [],
            "description": "",
            "createdAt": "2022-12-22 08:51:39",
            "modifiedAt": "2022-12-28 15:44:19",
            "private": false,
            "type": [
                "Description Image"
            ],
            "size": 55,
            "sortOrder": null,
            "title": "Einbau- und Betriebshinweise",

            "fileId": "63ac0d0cde8995351",
            "fileName": "Einbauhinweis_1380_1390.png",
            "filePathsData": {
                "download": "upload/files/7k4dp/Einbauhinweis_1380_1390.png",
                "thumbs": {
                    "small": "upload/thumbnails//16801/small/Einbauhinweis_1380_1390.png",
                    "medium": "upload/thumbnails//16801/medium/Einbauhinweis_1380_1390.png",
                    "large": "upload/thumbnails//16801/large/Einbauhinweis_1380_1390.png"
                }
            }
        }
    ]
}

This is gone with 1.9.21+ instead, one can use /Product/:entityId/productAssets which returns an entirely different collection, that includes prefixed product_* and asset_* data.

{
    "total": 6,
    "list": [
        //...
        {
            "id": "314",
            "deleted": false,
            "isMainImage": false,
            "sorting": 10,
            "scope": "Global",
            "tags": null,
            "createdAt": null,
            "modifiedAt": null,
            "fileId": "63ac0d0cde8995351",
            "fileName": "Einbauhinweis_1380_1390.png",
            "filePathsData": {
                //..
            },
            "icon": null,
            "product_name": null,
            "product_sku": "TEST1380PR01",
            "product_isActive": true,
            "product_amount": 0,
            "product_price": null,
            "product_productStatus": "draft",
            "product_longDescription": null,
            "product_sortOrder": 110,
            "product_hierarchySortOrder": null,
            "product_data": null,
            "product_createdAt": "2022-12-29 08:44:14",
            "product_modifiedAt": "2023-08-01 08:41:33",
            "product_sorting": null,
            "product_materialcode": "TEST1380",
            "product_additionalInformation": null,
            "product_shortDescription": null,
            "product_packingnotice": null,
            "product_priceCurrency": null,
            "asset_preview": null,
            "asset_icon": null,
            "asset_url": "/upload/files/7k4dp/Einbauhinweis_1380_1390.png",
            "asset_afterSaveMessage": null,
            "asset_height": 1470,
            "asset_width": 1500,
            "asset_colorSpace": "SRGB",
            "asset_colorDepth": "8",
            "asset_orientation": "Landscape",
            "asset_isActive": true,
            "asset_tags": null,
            "asset_name": "Einbauhinweis_1380_1390.png",
            "asset_description": null,
            "asset_createdAt": "2022-12-22 08:51:39",
            "asset_modifiedAt": "2022-12-28 15:44:19",
            "asset_private": false,
            "asset_type": [
                "5fbe2b489bf7238b3"
            ],
            "asset_size": 55,
            "asset_sortOrder": null,
            "asset_hierarchySortOrder": null,
            "asset_title": null,
            "asset_lang": "de",
            "asset_isRoot": true,
            "asset_hasChildren": true,
            "asset_hierarchyRoute": [],
            "asset_inheritedFields": null,
            "productId": "63ad535edbc9ad1",
            "productName": "Folienringe",
            "assetId": "63a41a9be351041b6",
            "assetName": "Einbauhinweis_1380_1390.png",
            "channelId": null,
            "channelName": null,
            "createdById": null,
            "createdByName": null,
            "modifiedById": null,
            "modifiedByName": null,
            "product_brandId": null,
            "product_brandName": null,
            "product_taxId": null,
            "product_taxName": null,
            "product_packagingId": "640f0db75bdfb",
            "product_productSerieId": null,
            "product_productSerieName": null,
            "product_catalogId": "5ff47082272d74ecd",
            "product_catalogName": "Lieferprogramm (Katalog)",
            "product_createdById": "1",
            "product_createdByName": "Admin",
            "product_modifiedById": "1",
            "product_modifiedByName": "Admin",
            "product_ownerUserId": "1",
            "product_ownerUserName": "Admin",
            "product_assignedUserId": "1",
            "product_assignedUserName": "Admin",
            "product_mainImageId": null,
            "product_mainImageName": null,
            "product_profileId": "61fc4387732815b9f",
            "product_profileName": "PR01",
            "asset_libraryId": null,
            "asset_libraryName": null,
            "asset_createdById": "62bdb456d7cb",
            "asset_createdByName": "",
            "asset_modifiedById": "1",
            "asset_modifiedByName": "Admin",
            "asset_fileId": "63ac0d0cde8995351",
            "asset_fileName": "Einbauhinweis_1380_1390.png",
            "asset_filePathsData": {
                //...
            }
        }
    ]
}

This is a breaking change for me, since the endpoint and the return object are fundamentally different, and it wasn't documented anywhere.

The Quick Fix

I was able to replace /Product/:entityId/assets with /Asset?where[0][type]=linkedWith&where[0][attribute]=productAssets_product&where[0][value][0]={{entityId}} which returns the expected output.

{
    "total": 6,
    "list": [
        {
            "id": "63a41a9be351041b6",
            "name": "Einbauhinweis_1380_1390.png",
            "deleted": false,
            "icon": null,
            "url": "/upload/files/7k4dp/Einbauhinweis_1380_1390.png",
            "height": 1470,
            "width": 1500,
            "colorSpace": "SRGB",
            "colorDepth": "8",
            "orientation": "Landscape",
            "isActive": true,
            "tags": [],
            "description": "",
            "createdAt": "2022-12-22 08:51:39",
            "modifiedAt": "2022-12-28 15:44:19",
            "private": false,
            "type": [
                "Description Image"
            ],
            "size": 55,
            "sortOrder": null,
            "title": "Einbau- und Betriebshinweise",

            "fileId": "63ac0d0cde8995351",
            "fileName": "Einbauhinweis_1380_1390.png",
            "filePathsData": {
                //..
            }
        }
    ]
}

The new /Product/:entityId/productAssets call (and the /ProductAsset entity) ties assets closer to the PIM and helps to have basic product information included, which might reduce requests under certain circumstances. I'm sure the Atro devs have a reason for this decision, to me, it seems quite redundant, at least how I use it.

At the moment (2023-08-19), there is also a bug in the localization of asset_* fields https://github.com/atrocore/atropim/issues/524

My Email Game (Update)

Sat, 29 Jul 2023 00:00:00 +0000

I love email. It was one of the first things I used when I got online back in 2000, and it's still here. It just works.

But a lot in email gets me headaches and that is always mail client specific, like:

Interrupted threads when someone started a new subject to discuss a previous topic, and the inability of mail clients to add a mail to an existing thread.
HTML email is a pain in the rear and even worse is the garbage that every HTML mail editor adds (I'm looking at you Microsoft Office/Outlook however you call yourself these days)
Email clients that can't read my mind what's important and what not (classify emails).
No native markdown to HTML converter (given if you use basic Markdown formatting in text-only emails, the client might render it bold or italic, but no headlines or anything).
Adding notes to email (without sending mails to myself, but I guess that's the way to do)

So far, my journey included the following email clients:

Outlook Express did its thin, butt once I tried Eudora, there was no going back unless I had to.
With the demise of Eudora I tried BAT! mail, an incredible powerful and ugly mail client and Pegasus Mail - I can't remember it wasn't good.
Mozilla Thunderbird entered the scene and everything was possible. Extensions for every use case and active development.
I used Thunderbird exclusively and over time I missed some features like the two row mail listing, that was never planned to change and other features were just hacky (thread view)
Postbox claimed to solve my problems and it was based on Thunderbird, so it was compatible with Thunderbird formats and it looked fresh. The first time I spend money on a mail client.
Sometimes I opened Gmail but never got used to it, until Googles Experiment Mailbox came on the scene. That was fantastic. Especially the classification of mails helped to deal with large amout of mail and promo mails.
Webinterfaces never felt right, Mailbox was shut down, Gmail sucks, Postbox was still there, but facing a difficult decision, because Thunderbird was about to be rebuild and only supporting their new webextensions.
Postbox closed their interface to use extensions, so a whole world of functionality I got used to disappear. "Send later", "Snooze Email" and the "Markdown Here" extension. Big bummer, but I kept using Postbox because I paid for it and it was still nice.
Today Postbox allows extensions again, but there are no compatible Thunderbird extensions anymore and no one cares about making them compatible. Not even Postbox. Markdown-Here got an update by the community, and it barely works. New features appear in Postbox sporadically, but nothing earth-shattering. Instead, they try to sell themes. Who cares?
I gave eM Client and Mailbird a shot. Mailbird is some garbage subscription Electron app. No thanks. eM Client impressed me mildly but suddenly I miss some Postbox features (RSS, complex filter, inline reply) I got used to and it wants another 50 to 70 EUR for a license.
Update 2023-07-29: I'm using my Office 365 address with Outlook Web. I have to say, it's working well so far. I even setup the native Microsoft Mail (I just wanted to connect my calendar) and it's not too bad either for just email.
For Bulk Email I started to use Betterbird, a Thunderbird fork with some fixes and some things that make it better, according to the developer. I can't tell what it is, but it's definitely not the UI/layout of the app.
Thunderbird announced to overhaul their ancient mail client and with Thunderbird Supernova they delivered the first version of their modernized email client. It has a multiline message list, and that sold it to me. I'm in the process of finally leaving Postbox and enjoying the integration of LanguageTool, Mailmerge, Markdown-Here.

eM Client as alternative?

eM Client has a working calender and address book. Postbox had Lightning in the past, but they got rid of it. I used the Sunlight calendar for a while but it got bought by Microsoft and then was shut down. The Microsoft calendar is not bad, but it's unreliable with WebDav calendar on Nextcloud.

eM Client impressed me with their thoughtful features, like showing a history of mails from the sender in an active mail, build in calendar and addressbook, notes and even chat (uses some chat protocols that I don't use, so there is that).

Now, the HTML editor in eM Client is just a basic textfield. They didn't even bother to use a monotype font. So it's hard to create html mail in there or signatures. It has wysiwyg tools though, but I don't trust them.

A thing I always ignored in email is encryption. I think the fact that no normal person is able to understand and setup PGP keys left this feature unused by 98% of mail users.

I've never received an encrypted email.

eM Client seems to make the setup easy, but I believe if nobody is using, why bother.

The filter capabilities in eM Client seem to be inferior to Postbox/Thunderbird. No time based filter (delete messages older than x days).

Another downside and a show stopper is the absence of an update of the "unread Mails" count on IMAP subfolders, according to this forum post: No updates or notifications from subfolders? - #55 by curious - Mail - eM Client. That's bad and I guess won't make me using it.

Other alternatives?

I can think of two: Hey Mail from the creators of Basecamp. It's a paid browser based client that seems to become very popular and I guess you have to use their hey.com mail address.

And there is Plum Mail which looks like a web client with some nice helpers, to pin mails and make notes and keep a conversation together. Subscription based and in beta right now. So I'll pass again.

Email tools I use or wish that existed

1. Classification of mails

Every mail client has a trainable spam filter. You would think that this technology could be implemented to train the algo to classify other topics? Nope. Nowhere to find in offline mail clients. That's only offered by cloud based mail systems like Gmail, Hey etc.

But there is Popfile - it's an ancient piece of software, last updated 2015 that sits between your mail client and the mailserver and analyzes mails and sort it into folders. It comes with a webinterface to train your mail and just works fine. Only thing to criticize is it works with only one IMAP connection. That's a bummer, but for my main mailbox it kind of works well to differenciate between notification mail, spam and important stuff.

2. Merging loose mail threads

There was a Thunderbird extension that aimed to do that, but as far as I know, mail threads are identified by a message-id and I think it's up to the mail client to be lax and group by subject line or whatever. Oh well.

3. Markdown

It seems like such a niche feature that probably is easy to implement (especially for Postbox, since there is an existing extension they just need to fix), but so far there is only one MacOS client that offers Markdown out of the box, so I won't hold my breath that eM Client or any other client will see that feature ever. (There is Markdown-Here Revival which works nicely)

4. Procmail filter editor in client

If I setup a local email filter I would love to see that reflected optionally as procmail filter on the server. But I guess that's too special and works only if your provider gives you access or an interface to that (as all-inkl.com does) combined with a Bayes filter like Popfile - that would be perfect.

5. Exchange support

I have to use at least one Exchange mail account and I don't want to know too much about this weird Microsoft product, just so much, it is not easy to setup if you don't use Outlook or some other MS software, especially if an ancient version of Exchange is used. I have to run a little proxy called DavMail that sits between Postbox or eM Client and deals with Exchange server. Works alright, but y tho?

Update: with Office 365 it's better integrated, until they changed something with their IMAP/SMTP connections. Either I use Outlook Web or in Thunderbid I now have to use this Owl extension because Thunderbird or MS messes up Unicode characters and umlauts when sending via SMTP. (Ä turns to ?). Y tho?

Encrypted chat via email

A well setup mail client is one thing, but what if you could use email as chat app? There is the Delta Chat messenger app, that connects to your email account, and offers you a nice chat interface as you know it from Signal or WhatsApp or Skype. Messages are simple emails that your chat partner can read in their email client. And if they use Delta Chat as well, it's even better. And if both partys youse Delta Chat, then they can use the Autoencrypt feature so messages are only readable by the client and nowhere else.

This is a really privacy friendly way to use a proven, robust and decentral infrastructure to communicate.

As far as I understand, this is not as secure messaging as Signal, because email metadata is not encrypted and therefore you leave traces. But I'm no expert here, I just think Delta Chat is a good thing to have.

tl;tr or conclusion

There isn't one. I guess I ~~keep working around the pain I have with Postbox and won't inflict pain in buying eM Client at this point and work around other issues there~~ switch back to Thunderbird, now that they have multiline message list (or cards as they call it).

As Chris Coyier states:

In fact, you come to love [email], because of how effective of a communication method it is: it’s public, it’s async, it can hold files, it can be of any length, it’s threaded.

Tooling isn’t needed to “fix” email. Email clients these days, for the most part, are interchangeably good.

Not a Big Deal – Email is Good

Web Components and Internationalization

Wed, 31 May 2023 00:00:00 +0000

Web Components and Internationalization

While exploring Web Components for a current project, came to a halt when faced with the question of language versions of a component.

As advertised, Web Components are portable pieces that can be inserted into any project. Nevertheless, Web Components can be linked directly to a project, but why would you use them instead of using Vue, React or any server-side template language? These are questions I still have to answer myself.

But back to my initial problem: What about the internationalization of Web Components?

I found a blog post¹ that goes much deeper, and from there I cobbled together my low effort version below.

The gist is, if you have a bunch of components with strings that can be translated, you handle them in the bundle and load external language files when the component starts up.

If you have a simple, single component, then you can add the language strings inside the component definition, query the document language <html lang="en"> or the navigator.language and be done with it. This is still not the most portable solution, but it works for that use case.

More complex implementation of that idea here: web-components/note-list.js at master · RolandDreger/web-components · GitHub

const FALLBACK_LANG = "en";

class MyBookmark extends HTMLElement {
  constructor() {
    super();
    this._saved = false;
  }
  getDocumentLang() {
        return (
            document.body.getAttribute("xml:lang") ||
            document.body.getAttribute("lang") ||
            document.documentElement.getAttribute("xml:lang") ||
            document.documentElement.getAttribute("lang") ||
            FALLBACK_LANG
        );
    }

  render() {
    const lang = this.getDocumentLang();
    const state = this._saved ? 'bookmarks.button.bookmarked' : 'bookmarks.button.bookmark';
    const messages = {
      'de': {
            'bookmarks.button.bookmark': 'merken',
            'bookmarks.button.bookmarked': 'gemerkt'
      },
      'en': {
            'bookmarks.button.bookmark': 'save',
            'bookmarks.button.bookmarked': 'saved'
      },
    };
    const html = `${messages[lang][state]}`;
  }
}

https://dev.to/btopro/reviewing-i18n-solutions-a10 ↩

Using Web Components and HTMX

Fri, 26 May 2023 00:00:00 +0000

Using Web Components and HTMX

A little excursion in using Web Components with and in HTMX. The results will... confuse you (or me).

I was wondering if Web Components are rendered automatically when they appear in an HTMX response, or if I need to "activate" them somehow. Yes, they are rendered when they are added to the DOM, which is great. Simply send them, and they will take care of themselves.

In the following Codepen demo, I reused my HTMX/Bootstrap tabs demo and return two custom HTML elements. First Paul Irish's lite-youtube-embed and second a custom element that renders a list of my Github Gists.

<lite-youtube videoid="RgA81-HvXVE" playlabel="Play"></lite-youtube>

<mo-gist username="marcus-at-localhost"></mo-gist>

See the Pen HTMX Bootstrap Tabs (Extension) with Web Components by Marcus at Localhost (@localhorst ) on CodePen.

I guess the next step is to figure out when not to use Web Components, what's up with the shadow DOM, and whether it's all worth it. (see the update below)

When I consider the previous usage, the lite YouTube embed is a perfect fit for Web Components. A single element that improves HTML (rather than using <iframe>).

On the other hand, my list of the most recent GitHub Gists is doing too much and is possibly not flexible enough. In this case, I treat the Web Component more like a template engine.

Should Web Components be used only for portable bits of HTML that can be used in another context? Can they be linked to my project without?

I'm still undecided.

Update 2023-06-01

I was building a Web Component to reduce noise in the rendered page.

Noise in terms of having this:

<mo-bookmark id="12345678" saved="true">

compared to this:

<button id="btn_bookmark_item_463556218" class="btn btn-light p-0 border-0 d-inline-flex align-items-center justify-content-center" hx-delete="/bookmark/61f7e4a8b3c66bcf4">
    <span class="fs-8 text-uppercase mx-2">saved</span>
    <span class="fs-6 bg-dark text-white d-flex align-items-center justify-content-center" style="width:1.75rem;height:1.75rem">
        <svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 32 32"><path fill="currentColor" d="M24 2H8a2 2 0 0 0-2 2v26l10-5.054L26 30V4a2 2 0 0 0-2-2Z"></path></svg>
        <span class="htmx-indicator fs-7 spinner-border spinner-border-sm" role="status" aria-hidden="true"></span>
    </span>
</button>

...several times on the page.

All the styling and functionality will be implemented in JavaScript and executed on the client-side.

However, how can you maintain the component framework-agnostic approach when using HTMX? I prefer not to include hx-* attributes within the component definition (although, why not?).

Then, I discovered a method to inject custom attributes into the component, like <mo-bookmark hx-post="/bookmark/1234" saved="true">. It seemed to work to some extent, but it led to additional complications that significantly increased the complexity. In the end, the effort to save a few kilobytes of redundant HTML wasn't worth it.

During my exploration, I came across two intriguing blog posts by Christopher Prohm. He created htmx Web Components, enabling reusability (Reusable htmx components with custom elements - Christopher Prohm), and even integrated htmx with React (Mixing React and htmx - Christopher Prohm), which is quite a bold move, given the ongoing trend to port React to htmx.

Taking it a step further, there are React components with HTMX available (reggi/htmx-components: 🧩 Async HTMX + JSX). Although I don't have experience with React, I find it amusing that HTMX aims to replace certain uses of React, only to potentially be absorbed by it.

Nevertheless, these are just my thoughts expressed spontaneously, and they may have deviated slightly from the main topic. However, I now have a clearer understanding, and it appears that integrating Web Components within HTMX works effectively. The real question is, do you truly require Web Components?

How to Load Content into a Bootstrap Offcanvas Component with HTMX and Save State as a Hash in the URL

Sat, 25 Mar 2023 00:00:00 +0000

How to Load Content into a Bootstrap Offcanvas Component with HTMX and Save State as a Hash in the URL

To open a Bootstrap Offcanvas Component and load some HTML fragment with HTMX, the first thing I tried was the following:

Equip the canvas (or modal) opener with the HTMX attributes
call it a day

<a class="btn btn-light" 
    href="/sidebar"
    hx-get="/sidebar" 
    hx-select=".bookmark-list" 
    hx-target=".offcanvas-body"
    data-bs-toggle="offcanvas"
    data-bs-target="#offcanvas">
    Open Sidebar
</a>

<div 
     id="offcanvas" 
     class="offcanvas offcanvas-start">
    <div class="offcanvas-body">
        Loading...
    </div>
</div>

This approach is completely decoupled; the click event triggers both the Bootstrap behavior and the HTMX ajax call. Both are unaware of one another.

If the canvas fails to open, the Ajax request may succeed, but the result cannot be viewed.

Back to the drawing board!

Connecting the behavior of HTMX and Bootstrap JS

The first thought is to use HTMX and listen for the 'htmx:load' event, then call .show() to open the Offcanvas component. That would necessitate some UI to indicate that the loading is complete before the canvas appears.

Or the other way around, listen to show.bs.offcanvas and then trigger htmx.ajax() to pull in the server-rendered HTML. This is better because it shows, something is happening right away.

Saving open state in the URL

That previous approach makes the button or link responsible for opening the canvas (or modal)?

If I navigate to /sidebar#offcanvas I want the sidebar to be open on page load with the HTMX ajax request triggered. The problem with this approach is, the button is the single source of truth that holds the URL that's getting loaded by HTMX via hx-get (or href).

I could go and htmx.find('a[href="/sidebar"]').href and then use that in the Ajax request. Or I trigger a click on the button, that triggers the behavior. But that seems weird and too tightly coupled.

document.addEventListener('DOMContentLoaded', function () {

    const el = '#offcanvas';
    let bookmarksOffcanvasInstance = bootstrap.Offcanvas.getOrCreateInstance(el);

    if(location.hash === '#offcanvas'){
        // open the sidebar
        bookmarksOffcanvasInstance.show();
        // find the button and make an
        // ajax call via 
        htmx.ajax('GET', htmx.find('a[href="/sidebar"]').href, {/* target etc */})

        // -- OR --

        htmx.trigger('a[href="/sidebar"]', "click");
    }
});

Make the Offcanvas component the single source of truth

There is another way I found by watching a video² about doing something similar with AlpineJS and an open issue with HTMX³ that brought me to the following solution:

1. The button or link to open the sidebar is only responsible for that

<a class="btn btn-light" 
    href="/sidebar"
    data-bs-toggle="offcanvas"
    data-bs-target="#offcanvas">
    Open Sidebar
</a>

2. Tie the HTMX logic to the canvas itself and trigger a HTMX custom event

<div 
     id="offcanvas" 
     class="offcanvas offcanvas-start"
     hx-get="/sidebar" 
     hx-select=".sidebar" 
     hx-target=".offcanvas-body"
     hx-trigger="filter-event">
    <div class="offcanvas-body">
        Loading...
    </div>
</div>
</div>

<script>
const el = document.getElementById("offcanvas");

// after the canvas was opened, trigger the hx-get with
// the custom event and add the url with the state of the canvas
// into the history
el.addEventListener('shown.bs.offcanvas', event => {
    htmx.trigger(event.target, "filter-event");
    history.pushState(null, null, '#' + event.target.id);
})

// on hiding the sidebar, remove the hash
el.addEventListener('hide.bs.offcanvas', event => {
    history.pushState("", document.title, window.location.pathname);
})
</script>

See the Pen Untitled by Marcus at Localhost (@localhorst) on CodePen.

Now, all the information for the Ajax request is associated with the offcanvas component, and it does not matter what triggers the opening of the sidebar; everything is contained in a single location.

My initial thought probably stems from my synchronous consideration of requests. A click alters the appearance of another element after a page reload.

With JavaScript's asynchronous nature, this behavior is out the window.

Bonus: Offcanvas activated by AlpineJS that triggers HTMX custom event

I adopted the whole script for the AlpineJS inline JavaScript style and here it is:

<a class="btn btn-light" href="#offcanvas">
    Open Sidebar
</a>
<div 
    id="offcanvas" 
    class="offcanvas offcanvas-start"

    x-data
    x-init="()=>{
        const oc = new bootstrap.Offcanvas('#offcanvas');
        if(location.hash === '#offcanvas') oc.show();
    }"
    @hashchange.window="if(location.hash === '#offcanvas') { bootstrap.Offcanvas.getOrCreateInstance(location.hash).show() }"
    @shown-bs-offcanvas.dot="
        htmx.trigger($event.target, 'filter-event');
        history.pushState(null, null, '#' + $event.target.id);"
    @hide-bs-offcanvas.dot="history.pushState('', document.title, window.location.pathname);"

    hx-get="/sidebar" 
    hx-select=".sidebar" 
    hx-target=".offcanvas-body"
    hx-trigger="filter-event">

        <div class="offcanvas-body">
            Loading...
        </div>
</div>

<template url="/sidebar" delay="1500">
    <h2>Sidebar Headline only visible when /sidebar is directly requested</h2>
  <div class="sidebar">
        Sidebar
    </div>
</template>

A working Codepen can be found under https://codepen.io/localhorst/pen/RwYvWyE (log in, switch to debug mode to see that URL hash change).

What I like about this approach is, it's very compact. Everything is in one place. No snippets here and bits there. You look at the markup of that component and that's all there is. At least for this demo. At the same time, it's ugly, hard to format and as complexity grows you'll end up putting stuff in a dedicated script block or so.

I don't know if there is any need to add AlpineJS in the mix as a third abstraction of code.

It's a matter of style and maintenance I guess. But now I know how to listen for events from Bootstrap components (see the .dot modifier) in AlpineJS.

If you have correction or thoughts about it, please let me know. I don't claim, that's the way to do it. I just made it work that way.

Notes and Links

Thu, 02 Feb 2023 00:00:00 +0000

Notes and Links

I'm still in search for a good title for this series, where I just post links. Maybe one day.

Bring back personal blogging - The Verge - I'm old, and I'm all nostalgic for Web 1.0 as well. Not really, but it was a good time to surf and build for the web in 2001.
CatGPT - chat with a cat.
CMS Trap - Max Chernyak - yeah, been there, done that! The article warns about the dangers of overcomplicating web applications by building content management systems that hinder the development of the actual content.

Use Bootstrap 5.x Tabs with HTMX

Sun, 15 Jan 2023 00:00:00 +0000

Use Bootstrap 5.x Tabs with HTMX

Yes, we all love HTMX because after years of rolling our eyes at React&co⁴, it's our time to shine with our knowledge from the good old days of server-rendered HTML, pulled in via jQuery ajax calls.

No need to introduce HTMX because you are probably here to find out how to hx-get HTML fragments into Bootstrap tabs.

The HTMX docs offer two ways to create tabs:

Tabs (Using HATEOAS) - the state of the tabs is being rendered on the server
Tabs (Using Hyperscript) - the state is managed by esoteric Hyperscript declarations.

Personally, I dislike both approaches and prefer to work within Bootstrap's constraints and use as many of their tools as possible. In this case for the tabbed interface, to control active state.

First I setup a really simple example and then I introduce a HTMX-Extension to avoid code duplication.

1. Basic Example

A typical Bootstrap .nav-link tab and corresponding .tab-pane looks like this: (I'll leave off all the attribute noise)

<nav>
<button class="nav-link" data-bs-toggle="tab" data-bs-target="#nav-home">Home</button>
</nav>
<div>
    <div class="tab-pane" id="nav-home">
        Home Content
    </div>
</div>

[data-bs-toggle="tab"] initializes the tabs, [data-bs-target="#nav-home"] points to .tab-pane#nav-home and .show.active said tab on click. Bootstrap takes care of the styles that reflect the state of the tabs, adds the .active class to the .nav-link and the .tab-pane and changes the state if you click another tab.

Now loading a remote html fragment into a .tab-pane could be done with a Bootstrap event:

const tabEl = document.querySelector('button[data-bs-toggle="tab"]')
tabEl.addEventListener('show.bs.tab', event => {
  // ajax call to the server, inject html into pane...
})

Instead we use HTMX. Here again the very simplified code example:

<nav>
<button class="nav-link" hx-get="/home" hx-target="#nav-home" data-bs-toggle="tab" data-bs-target="#nav-home">Home</button>
</nav>
<div>
    <div class="tab-pane" id="nav-home"></div>
</div>

The attribute [hx-get="/home"] is the link to an HTML fragment, and [hx-target="#nav-home"] tells HTMX in which pane the fragment will be injected.

This should all work out of the box. There might be some extra work to do, to smooth out CSS transitions. The tab will be activated by Bootstrap immediately, and stays blank until the HTML fragment was loaded!

See the Pen HTMX Bootstrap Tabs (Basic) by Marcus at Localhost (@localhorst) on CodePen.

To prevent the tab from being loaded from the server on every activation, I added this little snippet, which checks if there are HTML elements from a previous call already in the .tab-pane and if yes, then just prevent the request.

htmx.on("htmx:beforeRequest", function(evt) {
    if(evt.detail.target.hasChildNodes()){
        // prevent another request
        evt.preventDefault()
    }
});

2. HTMX Extension

To reduce code duplication, [hx-target] and [data-bs-target] point to the same .tab-pane [id], I just marked up the tabs with [data-bs-*] attributes if I'm not using [hx-*]

Here is a ultra-minimal example:

<script src="/bs-tabs.js"></script>
<div hx-ext="bs-tabs">
    <nav>
        <button class="nav-link" hx-get="/home" hx-target="#nav-home">Home</button>
    </nav>
    <div>
        <div class="tab-pane" id="nav-home"></div>
    </div>
</div>

HTMX extensions are typically pulled in via an external script file. With [hx-ext="bs-tabs"] I declare this part of the DOM as part of the extension.

Then I add the [hx-*] attributes and the extension takes care of the rest of activating the Bootstrap behavior. The comments in the following code should help you to understand what's going on.

(function(){
    htmx.defineExtension('bs-tabs', {
        onEvent: function (name, evt) {
            // before a request is made, 
            // check if there the content was already loaded into the tab pane
            if (name === "htmx:beforeRequest") {
                if(evt.detail.target.hasChildNodes()){
                    // stop request
                    return false;
                }
            }

            if (name === "htmx:afterProcessNode") {
                let allLinks = htmx.findAll(htmx.find('[hx-ext="bs-tabs"]'), '.nav-link');
                // loop through all .nav-links
                allLinks.forEach(triggerEl => {
                    // if the bootstrap data attribute for the target pane is missing...
                    if(!triggerEl.hasAttribute('data-bs-target')){
                        // ... add it. It's the same as the hx-target attribute.
                        triggerEl.setAttribute('data-bs-target', triggerEl.getAttribute('hx-target'));
                    }
                    // add the attribute that tells initializes the bootstrap functionality
                    triggerEl.setAttribute('data-bs-toggle', 'tab');
                });
            }
            return true;
        }
    });
})();

Here is a working example. Btw. the HTMX folks way to set up demos that mock remote calls is genius!

See the Pen HTMX Bootstrap Tabs (Extension) by Marcus at Localhost (@localhorst) on CodePen.

These are very simple examples. They don't address the state of the tabs if you need to load a fragment on page load. I might add this later.

That's it. Let me know if there is something wrong or a better way. I'm still wrapping my head around the whole JS events and states. Also I hope this inspires more posts and examples of HTMX being used with other frontend frameworks.

Here is an example how to use Bootstraps form validation styles with HTMX!

because we couldn't understand how it works, at least I didn't ↩