Bhasher/kill-the-newsletter
Bhasher
/
kill-the-newsletter
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Convert email newsletters into Atom feeds
170 Commits 1 Branch 0 Tags 2 MiB
TypeScript 67.9%
JavaScript 31.6%
Dockerfile 0.5%
Go to file
Leandro Facchinetti e452838d27 Don’t store alternates explicitly to save disk space 
Instead, fetch alternates from within the feed on the demand.

This makes alternates marginally more expensive to retrieve, but saves on storage (which we were running out on the DigitalOcean deployment), and is a cleaner architecture overall: no need to keep the feeds and alternates in sync.

Here’s a script to migrate existing feeds:

// Call me with, for example: env "BASE_URL=https://kill-the-newsletter.com" npx ts-node migrate.ts
// I’m idempotent and reentrant, you may call me multiple times if necessary (for example, if the migration fails in the middle for whatever reason)

import { promises as fs } from "fs";
import path from "path";
import { JSDOM } from "jsdom";

const BASE_URL = process.env.BASE_URL ?? "http://localhost:8000";
const FEEDS_PATH = "static/feeds";

(async () => {
  await fs.rmdir("static/alternate", { recursive: true });
  for (const feedPath of (await fs.readdir(FEEDS_PATH)).filter((feedPath) =>
    feedPath.endsWith(".xml")
  )) {
    const text = await fs.readFile(path.join(FEEDS_PATH, feedPath), "utf-8");
    const feed = new JSDOM(text, { contentType: "text/xml" });
    const document = feed.window.document;
    const feedIdentifier = document
      .querySelector("id")!
      .textContent!.split(":")[2];
    for (const entry of document.querySelectorAll("entry")) {
      const entryIdentifier = entry
        .querySelector("id")!
        .textContent!.split(":")[2];
      entry
        .querySelector(`link[rel="alternate"]`)
        ?.setAttribute(
          "href",
          `${BASE_URL}/alternate/${feedIdentifier}/${entryIdentifier}.html`
        );
    }
    await fs.writeFile(
      path.join(FEEDS_PATH, feedPath),
      `<?xml version="1.0" encoding="utf-8"?>${feed.serialize()}`.trim()
    );
    console.log(feedIdentifier);
  }
})();
 		2020-11-24 17:12:14 +00:00
.github/workflows . 2020-08-04 22:47:58 +01:00
.vscode . 2020-08-04 22:47:58 +01:00
static Don’t store alternates explicitly to save disk space 2020-11-24 17:12:14 +00:00
.dockerignore Changes to Dockerfile based on https://nodejs.org/en/docs/guides/nodejs-docker-webapp/ 2020-07-24 09:03:17 +01:00
.gitignore . 2020-08-05 12:32:16 +01:00
.prettierignore Remove www 2020-08-04 22:57:49 +01:00
CODE_OF_CONDUCT.md . 2020-03-17 21:05:16 -04:00
Caddyfile Remove www 2020-08-04 22:57:49 +01:00
Dockerfile . 2020-08-04 22:47:58 +01:00
LICENSE Remove www 2020-08-04 22:57:49 +01:00
README.md . 2020-09-13 02:34:26 +01:00
index.ts Don’t store alternates explicitly to save disk space 2020-11-24 17:12:14 +00:00
package-lock.json Bump dot-prop from 4.2.0 to 4.2.1 2020-10-30 20:46:04 +00:00
package.json Add html tagged template literal 2020-08-10 00:30:12 +01:00
test.ts Remove www 2020-08-04 22:57:49 +01:00
tsconfig.json Rmeove old line 2020-09-03 07:30:19 +01:00

README.md

Kill the Newsletter!

Convert email newsletters into Atom feeds

Convert email newsletters into Atom feeds

Source Continuous Integration

Watch the Code Review!

Deploy Your Own Instance (Self-Host)

  1. Create accounts on GitHub, Namecheap, and DigitalOcean.

  2. Fork this repository.

  3. Create a deployment SSH key pair:

    $ ssh-keygen

    Private key (id_rsa): Add to your fork under Settings > Secrets as a new secret called SSH_PRIVATE_KEY.

    Public key (id_rsa.pub): Add to your fork under Settings > Deploy keys and to your DigitalOcean account under Account > Security > SSH keys.

  4. Buy a domain on Namecheap.

  5. Create a DigitalOcean droplet:
    Image Ubuntu 18.04.3 (LTS) x64
    Plan Starter Standard $5/mo
    Additional options Monitoring
    Authentication Your Deployment SSH Key
    Hostname <YOUR DOMAIN, FOR EXAMPLE, “kill-the-newsletter.com”>
    Backups Enable

  6. Assign the new droplet a Firewall:
    Name <YOUR DOMAIN, FOR EXAMPLE, “kill-the-newsletter.com”>
    Inbound Rules ICMP
    SSH 22
    Custom 25 (SMTP)
    HTTP 80
    HTTPS 443

  7. Assign the new droplet a Floating IP.

  8. Configure the DNS in Namecheap:

    Type Host Value
    A @ <FLOATING IP>
    CNAME www <YOUR DOMAIN, FOR EXAMPLE, “kill-the-newsletter.com”>
    MX @ <YOUR DOMAIN, FOR EXAMPLE, “kill-the-newsletter.com”>

  9. Configure the deployment on package.json, particularly under the following keys:

    • apps.env.BASE_URL.
    • apps.env.EMAIL_DOMAIN.
    • apps.env.ISSUE_REPORT.
    • deploy.production.host.
    • deploy.production.repo.

  10. Configure Caddy, the reverse proxy, on Caddyfile.

  11. Setup the server:

    $ ssh-add
$ npm run deploy:setup

  12. Migrate the existing feeds (if any):

    $ ssh-add
$ ssh -A root@<YOUR DOMAIN, FOR EXAMPLE, “kill-the-newsletter.com”>
root@<YOUR DOMAIN, FOR EXAMPLE, “kill-the-newsletter.com”> $ rsync -av <path-to-previous-feeds> /root/kill-the-newsletter.com/current/static/feeds/
root@<YOUR DOMAIN, FOR EXAMPLE, “kill-the-newsletter.com”> $ rsync -av <path-to-previous-alternate> /root/kill-the-newsletter.com/current/static/alternate/

  13. Push to your fork, which will trigger the GitHub Action that deploys the code and starts the server.

Run Locally

Install Node.js and run:

$ npm install
$ npm run develop

The web server will be running at http://localhost:8000 and the email server at smtp://localhost:2525.

Run Tests

Install Node.js and run:

$ npm install-test

Docker Support (Experimental)

Install Docker and run:

$ docker build -t kill-the-newsletter .
$ docker run kill-the-newsletter

The web server will be running at http://localhost:8000 and the email server at smtp://localhost:2525.

For use in production, start with the example Dockerfile.