Add content index: keywords, tags/categories, search, and XML sitemap

Ships Blog tags/categories, Internal search, and XML sitemap together as
one shared mechanism, plus a new meta keywords request, rather than three
separate ones - the "worth deciding together" call ISSUES.md made when
XML sitemap was first written.

Content stays in files. Per-page metadata is four Twig blocks in the
root layout, the same override mechanism already used for
title/description/og_* - keywords (rendered), tags/changefreq/priority
(not rendered, harvested only). App\ContentIndexer crawls every routable
page (Overlay::listPageDirs(), new) and pulls each block via
Renderer::renderForIndex() (new) calling Twig's own renderBlock() API,
not regex-parsing .twig source, so overrides and layout inheritance
resolve exactly like a real render. Rendered HTML is stripped and
indexed into a SQLite FTS5 table for search.

Off by default (content_index_enabled) - same posture as Matomo/admin
auth, since this is a real SQLite dependency plenty of sites won't want.
Verified true zero footprint when disabled: no data/novaconium.sqlite
gets created, all three consumer routes 404 like they don't exist.
content_index_auto (default true) reindexes lazily on first stale touch
of a consumer route, never on a normal page view; both that path and the
explicit `php novaconium/bin/index-content.php` share one reindex().

Two real bugs caught by testing, not review: a reentrancy bug where
/search's own sidecar calling ensureFresh() during the crawl triggered a
nested reindex() mid-transaction (fixed with a static re-entrancy guard),
and a wrong PDO constant in the search sidecar. Also fixed two unrelated
pre-existing bugs found while building this: an unescaped {{ }} in
admin/docs/sidecars that fataled Twig on any real render of that page,
and a stale "no database layer yet" claim there and in Lib\Input's
docblock, left over from before Lib\Db shipped.

Lib\Db's migrations_dir now accepts an ordered list of roots, not just
one path, so the content index's schema could ship as a framework
migration (novaconium/migrations/) without colliding with project
migrations in App/migrations/ - the two-root extension point flagged in
AGENTS.md when SQLite groundwork shipped. Migrations are tracked by path
relative to the repo root rather than bare filename so two roots with a
same-named file can't shadow each other.

New consumer routes: novaconium/pages/sitemap.xml/ and
novaconium/pages/search/ (framework defaults), App/pages/blog/tag/[tag]/
(project-owned, since blog/ is project content - the existing hand
-written post array in App/pages/blog/index.php is untouched). Added
tags to the 4 existing blog posts as a real demonstration.

Closes the Blog tags/categories, Internal search, and XML sitemap
backlog items in novaconium/ISSUES.md.
This commit is contained in:
code
2026-07-14 17:35:30 +00:00
parent 4862526fa1
commit 37b6764431
30 changed files with 1123 additions and 157 deletions
+9
View File
@@ -42,4 +42,13 @@ return [
// set to actually gate anything; open access otherwise, same as the
// rest of /admin/*.
// 'draft_routes' => ['blog/upcoming-post'],
// Docs: /admin/docs/content-index — powers /sitemap.xml, /search, and
// blog tag browsing. Off by default (depends on SQLite); enable with:
// 'content_index_enabled' => true,
//
// Or enable but skip the automatic lazy reindex, relying only on
// `php novaconium/bin/index-content.php` (e.g. from a deploy step):
// 'content_index_enabled' => true,
// 'content_index_auto' => false,
];
+1
View File
@@ -6,6 +6,7 @@
{% block description %}The first post on this blog — a plain sidecar-less page, like every other post here now.{% endblock %}
{% block robots %}index, follow{% endblock %}
{% block tags %}welcome, meta{% endblock %}
{% block canonical %}{{ request_path|default('/') }}{% endblock %}
{% block og_type %}article{% endblock %}
+1
View File
@@ -6,6 +6,7 @@
{% block description %}A second post at its own URL, showing that adding a new page under App/pages/blog/ needs nothing but a new directory.{% endblock %}
{% block robots %}index, follow{% endblock %}
{% block tags %}meta{% endblock %}
{% block canonical %}{{ request_path|default('/') }}{% endblock %}
{% block og_type %}article{% endblock %}
+1
View File
@@ -6,6 +6,7 @@
{% block description %}A showcase of this theme's default styling for headings, lists, tables, code, and other common HTML elements.{% endblock %}
{% block robots %}index, follow{% endblock %}
{% block tags %}css, reference{% endblock %}
{% block canonical %}{{ request_path|default('/') }}{% endblock %}
{% block og_type %}article{% endblock %}
+65
View File
@@ -0,0 +1,65 @@
<?php
// blog/tag/[tag]/ — the [param] segment captures anything after
// /blog/tag/ into $params['tag'] (see /admin/docs/routing). This page is
// project-owned (unlike sitemap.xml/search, which are framework defaults
// under novaconium/pages/) because blog/ itself is project content, not
// framework machinery.
use App\ContentIndexer;
use App\Response;
use Lib\Db;
// Same two-step config load bootstrap.php/bin scripts use — this sidecar
// isn't handed $config, so it loads its own copy to read
// content_index_enabled before touching Lib\Db at all.
$config = require __DIR__ . '/../../../../../novaconium/config.php';
$appConfigFile = __DIR__ . '/../../../../../App/config.php';
if (is_file($appConfigFile)) {
$config = array_merge($config, require $appConfigFile);
}
// Content index is off by default (depends on SQLite) — see
// /admin/docs/content-index. When it's off, this route must 404 exactly
// like a page that doesn't exist, and never construct a Lib\Db connection
// (which would otherwise create data/novaconium.sqlite just because this
// file exists, even on a site that never opted in).
if (!$config['content_index_enabled']) {
return Response::html('404 Not Found', 404);
}
// Lazy reindex-if-stale — a no-op on most requests (only actually
// reindexes when a page's source file changed since the last index). Also
// guards against reentrancy: ContentIndexer's own crawl renders every
// page, but it never renders *this* route, since blog/tag/[tag] is a
// wildcard directory and Overlay::listPageDirs() skips [param]-wildcard
// dirs entirely (concrete tag values aren't knowable without a data
// source — see ContentIndexer's docblock).
ContentIndexer::ensureFresh();
$tag = $params['tag'];
// content_tags is a derived index built from each post's own
// {% block tags %} (see /admin/docs/content-index) — it's populated
// entirely by ContentIndexer::reindex(), never written to directly here.
// The route LIKE '/blog/%' filter matters because content_tags isn't
// blog-specific — any page anywhere on the site can declare tags, so this
// scopes results to blog posts only, the same way App/pages/blog/index.php
// itself only ever lists blog posts.
$posts = Db::query(
'SELECT content_pages.route, content_pages.title, content_pages.description ' .
'FROM content_tags ' .
'JOIN content_pages ON content_pages.route = content_tags.route ' .
'WHERE content_tags.tag = ? ' .
"AND content_pages.route LIKE '/blog/%' " .
'ORDER BY content_pages.route',
[$tag]
)->fetchAll(PDO::FETCH_ASSOC);
// $posts is [] (not an error) when nothing matches — the twig template
// renders a plain "no posts tagged ..." message for that case, same as
// /search does for a query with no results.
return [
'tag' => $tag,
'posts' => $posts,
];
+24
View File
@@ -0,0 +1,24 @@
{% extends layout %}
{% import '_layout/icons.twig' as icons %}
{% block title %}Posts tagged &ldquo;{{ tag }}&rdquo;{% endblock %}
{% block description %}Blog posts tagged {{ tag }}.{% endblock %}
{% block robots %}noindex, follow{% endblock %}
{% block content %}
<h1 class="icon-heading">{{ icons.tag() }}Posts tagged &ldquo;{{ tag }}&rdquo;</h1>
{% if posts|length > 0 %}
<ul class="post-list">
{% for post in posts %}
<li>
<a href="{{ post.route }}">{{ post.title }}</a>
{% if post.description %}<p>{{ post.description }}</p>{% endif %}
</li>
{% endfor %}
</ul>
{% else %}
<p>No posts tagged &ldquo;{{ tag }}&rdquo;.</p>
{% endif %}
{% endblock %}
@@ -6,6 +6,7 @@
{% block description %}A tour of the Twig syntax used throughout this site — output, filters, control structures, inheritance, and a few gotchas.{% endblock %}
{% block robots %}index, follow{% endblock %}
{% block tags %}twig, reference{% endblock %}
{% block canonical %}{{ request_path|default('/') }}{% endblock %}
{% block og_type %}article{% endblock %}