Files
novaconium/novaconium/pages/search/index.php
T
code 37b6764431 Add content index: keywords, tags/categories, search, and XML sitemap
Ships Blog tags/categories, Internal search, and XML sitemap together as
one shared mechanism, plus a new meta keywords request, rather than three
separate ones - the "worth deciding together" call ISSUES.md made when
XML sitemap was first written.

Content stays in files. Per-page metadata is four Twig blocks in the
root layout, the same override mechanism already used for
title/description/og_* - keywords (rendered), tags/changefreq/priority
(not rendered, harvested only). App\ContentIndexer crawls every routable
page (Overlay::listPageDirs(), new) and pulls each block via
Renderer::renderForIndex() (new) calling Twig's own renderBlock() API,
not regex-parsing .twig source, so overrides and layout inheritance
resolve exactly like a real render. Rendered HTML is stripped and
indexed into a SQLite FTS5 table for search.

Off by default (content_index_enabled) - same posture as Matomo/admin
auth, since this is a real SQLite dependency plenty of sites won't want.
Verified true zero footprint when disabled: no data/novaconium.sqlite
gets created, all three consumer routes 404 like they don't exist.
content_index_auto (default true) reindexes lazily on first stale touch
of a consumer route, never on a normal page view; both that path and the
explicit `php novaconium/bin/index-content.php` share one reindex().

Two real bugs caught by testing, not review: a reentrancy bug where
/search's own sidecar calling ensureFresh() during the crawl triggered a
nested reindex() mid-transaction (fixed with a static re-entrancy guard),
and a wrong PDO constant in the search sidecar. Also fixed two unrelated
pre-existing bugs found while building this: an unescaped {{ }} in
admin/docs/sidecars that fataled Twig on any real render of that page,
and a stale "no database layer yet" claim there and in Lib\Input's
docblock, left over from before Lib\Db shipped.

Lib\Db's migrations_dir now accepts an ordered list of roots, not just
one path, so the content index's schema could ship as a framework
migration (novaconium/migrations/) without colliding with project
migrations in App/migrations/ - the two-root extension point flagged in
AGENTS.md when SQLite groundwork shipped. Migrations are tracked by path
relative to the repo root rather than bare filename so two roots with a
same-named file can't shadow each other.

New consumer routes: novaconium/pages/sitemap.xml/ and
novaconium/pages/search/ (framework defaults), App/pages/blog/tag/[tag]/
(project-owned, since blog/ is project content - the existing hand
-written post array in App/pages/blog/index.php is untouched). Added
tags to the 4 existing blog posts as a real demonstration.

Closes the Blog tags/categories, Internal search, and XML sitemap
backlog items in novaconium/ISSUES.md.
2026-07-14 17:35:30 +00:00

104 lines
4.8 KiB
PHP

<?php
// /search — a framework default (novaconium/pages/, not App/pages/) since
// full-text search over the whole site is generic machinery, not project
// content. Has both this sidecar and an index.twig, unlike sitemap.xml —
// it renders a real HTML page (a form plus results), not a bypass-Twig
// Response.
use App\ContentIndexer;
use App\Response;
use Lib\Db;
use Lib\Input;
// Same two-step config load bootstrap.php/bin scripts use — this sidecar
// isn't handed $config, so it loads its own copy to read
// content_index_enabled before touching Lib\Db at all.
$config = require __DIR__ . '/../../config.php';
$appConfigFile = __DIR__ . '/../../../App/config.php';
if (is_file($appConfigFile)) {
$config = array_merge($config, require $appConfigFile);
}
// Content index is off by default (depends on SQLite) — see
// /admin/docs/content-index. When it's off, this route must 404 exactly
// like a page that doesn't exist, and never construct a Lib\Db connection
// (which would otherwise create data/novaconium.sqlite just because this
// file exists, even on a site that never opted in).
if (!$config['content_index_enabled']) {
return Response::html('404 Not Found', 404);
}
// Input::get(), not $_GET directly — see /admin/docs/sidecars' "Form
// security" section. Only Lib\Input's HTML/script-injection cleaning
// matters here (this value never reaches SQL unparameterized either way,
// see the FTS5 escaping note below).
$query = trim((string) Input::get('q', ''));
$results = [];
// Bare /search (no ?q=) just shows the empty form — skip the query and
// the reindex-freshness check entirely, so landing on the page cold costs
// nothing beyond the normal page render.
if ($query !== '') {
// Lazy reindex-if-stale — a no-op on most requests (only actually
// reindexes when a page's source file changed since the last index).
// Deliberately guarded by $query !== '' rather than called
// unconditionally at the top of the file: ContentIndexer's own crawl
// renders every real page, including this one, so /search visiting
// itself with an empty query during a crawl must NOT trigger another
// reindex — ContentIndexer also has its own reentrancy guard for this
// (see its docblock), but not paying the freshness-check cost on
// every bare page load is a second, independent reason this call sits
// inside the if.
ContentIndexer::ensureFresh();
// Parameter binding prevents SQL injection, but the bound value is
// still parsed as its own FTS5 query-language expression, not a plain
// string — a literal " or an FTS operator in $query could otherwise
// throw a syntax error or search for something unintended. Wrapping it
// as a quoted phrase (doubling any embedded ") makes the whole query
// an FTS5 phrase-match literal, neutralizing that syntax entirely.
// Verified against a literal ", "*", "OR", and a "'; DROP TABLE ..."
// attempt — all return normal (possibly empty) results, no fatal, no
// effect on the database.
$ftsQuery = '"' . str_replace('"', '""', $query) . '"';
// content_search is the FTS5 virtual table ContentIndexer::reindex()
// populates with route/title/stripped-HTML body — "rank" is an FTS5
// built-in column (not one we define) giving relevance ordering, most
// relevant first. LIMIT 50 is just a sanity cap, not real pagination.
$results = Db::query(
'SELECT route, title FROM content_search WHERE content_search MATCH ? ORDER BY rank LIMIT 50',
[$ftsQuery]
)->fetchAll(PDO::FETCH_ASSOC);
// A second query rather than joining content_pages into the FTS5
// query directly — content_search is a virtual table, and mixing a
// real table JOIN into an FTS5 MATCH query is more fragile than doing
// the description lookup separately. IN (...) with one placeholder
// per route, built from the exact route set the first query returned
// — never string-interpolating $query or user input into this SQL,
// only the already-fetched, already-trusted route values.
if ($results !== []) {
$routes = array_column($results, 'route');
$placeholders = implode(',', array_fill(0, count($routes), '?'));
$descriptions = Db::query(
"SELECT route, description FROM content_pages WHERE route IN ({$placeholders})",
$routes
)->fetchAll(PDO::FETCH_KEY_PAIR);
foreach ($results as &$result) {
$result['description'] = $descriptions[$result['route']] ?? '';
}
unset($result); // break the foreach-by-reference alias, standard PHP gotcha
}
}
// $results stays [] (not an error) for a blank query or one with zero
// matches — the twig template only distinguishes those two cases by
// checking $query itself, not by any error flag.
return [
'query' => $query,
'results' => $results,
];