docs(import): folder import, sparse records and per-file views (improvise-lpzu, improvise-8jk6.2)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze

context/code/cli.md

@@ -369,3 +369,24 @@ non-zero and writes nothing. It is checked before the build rather than read
 off the build's error, so the listing is structured data and not a parsed
 sentence. `separate` uses `ImportPipeline::suggested_duplicate_category`, and
 there is deliberately no flag to name the category yet.
+
+## 12 — `import` takes folders, and the CLI only says which arguments are folders (improvise-lpzu)
+
+`improvise import` takes any mix of files and folders. A folder stands for its
+`*.csv` files, in name order, so `import dir/` is `import dir/*.csv`. The rules
+and their reasons are in `context/code/formula-and-io.md`, "A folder of CSVs is
+one sparse model, with a view per file". The CLI's part is the one question
+`Storage` cannot answer: whether a path is a folder. `get_import_data` asks
+`Path::is_dir` for each argument, hands the resulting `ImportPath`s to
+`expand_import_paths` over `PhysicalFs`, and from there takes the path a list
+of files always took. A folder with no CSV in it is refused before anything is
+read. The message names the folder, and the run exits non-zero and writes
+nothing.
+
+A merge now reaches the build as an `ImportPipeline` carrying the merged files'
+`sources`, rather than as a bare JSON value. Both `--no-wizard` and the TUI
+wizard (`run_tui` → `App::start_import`) therefore build the per-file views.
+The headless path matters most, because the code-maat pipeline
+(improvise-1xfb.5.1) runs there. `a_headless_import_of_a_folder_merges_its_csv_files_and_repeats_byte_for_byte`
+pins the views, the sparse Data lines and the byte-identical second run, with
+the first run's output sitting in the folder it reads.

context/code/formula-and-io.md

@@ -632,6 +632,88 @@ one proves nothing.
 backend rather than only a file (improvise-fuj); the path-taking wrappers remain
 for the native CLI.
 
+### A folder of CSVs is one sparse model, with a view per file (improvise-lpzu)
+
+`improvise import <folder>` exists for improvise-1xfb.5.1. code-maat writes one
+CSV per analysis into a directory, and the schemas differ: coupling has
+`entity,coupled,degree,average-revs`, age has `entity,age-months`, and
+abs-churn has `date,added,deleted,commits`. The user ruled (2026-09-25):
+"Records are already sparse: the model can just leave null/empty values for
+columns not in a given CSV file." So files with different headers merge into
+**one** model. It is not refused, and it is not split into one model per schema.
+
+**A folder argument means exactly its `*.csv` files.** `expand_import_paths`
+takes the arguments in order and replaces each folder with its `*.csv`
+entries. The extension check is case-insensitive, the listing is not recursive,
+and the order is `Storage::list`'s, which is sorted by contract. Then the
+existing path runs unchanged: two or more files merge through `merge_csvs_from`
+with a `File` category, and one file imports alone. So `import dir/` is
+`import dir/*.csv`, including when the folder holds a single CSV. Four cases
+follow from that equivalence rather than being chosen one by one:
+
+- **Non-CSV entries are skipped**, not refused, because a glob skips them. This
+  includes the output `.improv`, which a second run finds in the folder.
+- **A folder with no CSV at all is refused**, naming the folder. A glob that
+  matches nothing is an error too, and importing nothing would write an empty
+  document.
+- **Files and folders mix freely**, in argument order.
+- **A file named twice is read twice.** For example, `dir/a.csv dir/` reads
+  `a.csv` twice, and its records collide with themselves. The headless path
+  refuses that like any other collision. Nothing deduplicates, because the
+  shell glob would not either.
+
+The listing goes through `Storage::list` so the browser host can reuse it
+(gotcha 2). `Storage` knows documents, not folders, so **whether an argument
+is a folder is the caller's question**. The CLI answers it at the edge with
+`Path::is_dir` and passes an `ImportPath::{File, Folder}`. An OPFS host would
+answer it from its own UI.
+
+**Sparse merge.** A record from a file without some column has no key for it,
+so `Plan::place` gives it no coordinate there (improvise-8jk6.2 above), and no
+measure cell. Nothing is fabricated: no `0` and no `""`. The analyzer already
+classified each column over the values present (improvise-l1f5), so a column
+only one file has is judged by that file's values alone. The category cap
+counts the **union** of every file's categories. Rows from different files
+cannot collide, because `File` differs.
+
+**Each file gets a view** (the user, 2026-09-25: "dir import should do this:
+one view filtered by the file column and projected to the columns in that
+file"). `merge_csvs_from` returns a `Merged` whose `sources` list each `File`
+item with the columns its header named. Two files with one stem share an item,
+so their columns are unioned. The pipeline carries `sources`, and the build
+adds one view per source after the merged `Default` view, which stays the
+initial view. Each per-file view:
+
+- **Is named after the file's stem.** A name already taken, including the
+  virtual `_Records` and `_Drill`, gets ` 2`, ` 3` and so on. A view named
+  `_Drill` would never be written and would be dropped on load.
+- **Puts `File` on `Axis::Filter`, fixed at that file's item.** Filter and
+  not Page, because the view *is* that file's projection. Paging to another
+  file would show that file's records through this file's columns and hidden
+  measures, which is wrong on every page but one. Filter constrains without
+  being navigable (gotcha 27). Until now it appeared only in the never-saved
+  `_Drill` view, but the `.improv` grammar has always round-tripped it.
+  `per_file_views_survive_a_round_trip_and_repeat_byte_for_byte` proves that on
+  a saved per-file view, and `a_saved_per_file_view_projects_only_its_file`
+  proves the shared `grid_in` projection, which both the TUI and the browser
+  draw from, carries it as a fixed coordinate with nothing to cycle.
+- **Seats the file's own categories** (its columns, plus date parts derived
+  from them) through `View::on_category_added`, in model order: the rule a
+  single-file import uses. Every other category goes on `None`. The file's
+  records lack those, so nothing splits.
+- **Puts `_Measure` on Page**, as `Workbook::new` does, **with other files'
+  measures hidden** and the page selection on the file's first measure. A
+  formula target comes from no file, so it stays visible.
+
+A per-file view is made only while `File` is an accepted category. A user who
+skipped or reclassified it has no filter to build views from.
+
+Two things this does not do. Page cycling still steps onto a hidden item and
+draws an empty grid: `[`/`]` in a per-file view can reach another file's
+measure (improvise-yd2t; the Default view already behaved this way). And a
+CSV with its own `File` column has it overwritten by the merge's, which is the
+existing behaviour of `merge_csvs`.
+
 ### The wizard's back path is a history stack
 
 `advance` pushes the step it leaves; `previous` pops it. Keeping the path
@@ -658,10 +740,33 @@ and a blank string are **absences, not names**; stringifying them used to invent
 an item literally called "null" (improvise-ych). Every other value keeps its
 printed form.
 
-A record missing a value for an accepted Category field has no place in the grid
-and is dropped; `build_model_with_warnings` counts it against the *first*
-category it is missing and reports the total, because without a warning that
-loss is invisible in a thousand-row import (improvise-5j9).
+**A record with no value for a category keeps its cell, without that
+coordinate** (improvise-8jk6.2). Records are sparse: a `CellKey` names only the
+coordinates it has. So a missing field, a `null` and a blank string all mean
+"no coordinate here". None of them makes an item: not `"null"`, not `""`, and
+no synthetic placeholder. The same rule covers labels, which used to become an
+item named `""`. The user's words, 2026-09-25: "this limit in import should be
+fixed, it dates to when this wasn't sparse." That limit was improvise-5j9's
+rule: such a record "has no place in the grid" and was dropped, counted and
+reported as `Skipped N of M records missing category values`. 5j9's warning
+made the loss visible, and the loss is now gone, so the warning went with it.
+The only record still skipped is one that is not a JSON object.
+
+Where such a record shows up is decided by the axes, not by the import. The
+test is `a_record_with_no_coordinate_is_drawn_only_where_that_category_is_off_the_grid`:
+
+- **The category on Row or Column:** no position holds the record. Its other
+  coordinates may still draw a row, but the cell under the category's items is
+  empty, and the column total leaves it out.
+- **The category on Page:** no selection shows it.
+- **The category on None:** it is aggregated like any other record.
+- **Records mode:** its row shows a blank cell in that category's column.
+- **`.improv`:** its Data line omits the category, and it round-trips.
+- **Drill:** a drill that does not constrain the category reaches it.
+
+Whether Row/Column/Page should give such records a place of their own (a
+"(none)" position) is **open, and it is the user's call**. It is raised on the
+lpzu PR. Nothing here invents one.
 
 ### Colliding records are a question, not a default (improvise-cgqt)
 
@@ -678,7 +783,9 @@ made it choose too. It chose `Refuse` and points at `:import`.
 
 **Detection comes before the build and shares the build's placement.** Where a
 record lands is `Plan::place`: its categories, then its date parts, then its
-labels, then its measure values, or a reason it is dropped. `collisions()` and
+labels, then its measure values. Only a record that is not an object is
+dropped, since a missing value is no coordinate rather than a reason to drop
+(improvise-8jk6.2). `collisions()` and
 `build_model_with_warnings` both call it, so the question the wizard asks and
 the thing the build does cannot disagree. Neither has a second walk of the
 record. **A collision is a cell that more than one record writes**, meaning a