[welmac-org] More storage for the WelMac web site to use

[David Empson]

I’ve been experimenting with Backblaze B2 cloud storage and have created an account there for president at welmac.org.nz, with no payment method so we are limited to its free tier. (On reflection it should have been associated with webmaster rather than president, which I can change later.)

For an overview of Backblaze B2, see https://www.backblaze.com/b2/cloud-storage.html

This gives WelMac up to 10 GB of free online storage as a place to store files that can be linked from our web site, rather than having to accumulate large files on our site (currently limited to 500 MB storage, and we want to reduce it). Once we have got the web site sufficiently cleaned up, this could allow a significant reduction in the amount of storage we need on the main web hosting service operated by Jo on our behalf.

Backblaze B2 limits free downloads to 1 GB per day (where days start at midnight UTC, i.e. either 12 PM or 1 PM NZ time depending on daylight saving) but has unlimited free uploads. A few people going crazy and downloading everything could cause a temporary denial of service to other users.

Should we ever grow to the point we need to start paying, storage above the free tier costs US0.5c per GB per month, and downloads above the free tier cost US2c per GB. This would require linking to someone's credit card but a cap can be set to limit the maximum cost.


As a first cut, I’ve created a public bucket in our Backblaze account called “welmac-public”, and uploaded 2016 and earlier back issues of Capital Apple as far back as we did the magazine as a PDF (March 2004), using a name prefix (simulated folder) of “capital_apple/" to match the web site. Total size is almost 190 MB (1.9% used, 98.1% free). I have created a first draft index to access them here:

http://welmac.org.nz/capital_apple/index2.php

The current year’s files are hosted on our site with the link style in the previous format we were using. All earlier years (in the table) comes from Backblaze. I haven’t bothered to recreate the tooltips indicating file sizes but they could be added with another editing pass.

Some minor cleanup of the table to make it easier to read would be good. We’d probably want to add a mechanism to at least count downloads of each issue.

For comparison with the old method, see this page, which has the most recent 22 issues (two years worth) hosted on our web site:

http://welmac.org.nz/capital_apple/

My plan would be to archive an entire year at a time once we start a new year, and only keep the current year’s issues on our web site. If we don’t expect to hit the 1 GB daily download limit I could just put new issues on Backblaze rather than our site, but I’d still like the links for the current year to stand out from the archives for previous years.

Once we’re happy with the appearance of the index, I’ll move it to replace the main index file so the back issues will be available to everyone. Creating a content index can be left as an exercise for later.

In order to make the filenames not produce somewhat mangled URLs, I renamed all the back issues to replace spaces in their names with hyphens, e.g. “Capital Apple 2016-01 Jan.pdf” became “Capital-Apple-2016-01-Jan.pdf” (otherwise spaces end up being converted to + signs upon upload to Backblaze, and that looks horrible). The name change will break alphabetic sorting for existing collections, but if the whole set is online there is less need for people to keep back issues.

In addition to Capital Apple, I’m thinking we can also stick meeting presentations on Backblaze. (They are currently in a folder on our site with no links to them apart from ones I publish in my article in Capital Apple.)

In principle this could be rewritten to generate the table dynamically using PHP to sign in to Backblaze via their API and get a directory listing, but that is also a potentially chargeable item if it is done often enough in one day. As it stands, adding a new row to the table for a new year is a copy and paste of a block of text plus find/replace, which is easy to manage. Adding file sizes in tooltips would be a bit more work but I can probably automate it.

Comments?

-- 
David Empson
dempson at actrix.gen.nz

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nzmac.org/mailman/private/welmac-org_nzmac.org/attachments/20170825/b6ae24b9/attachment-0004.html>