fix(import): a header-only or measureless file keeps its identity and a view of its own (improvise-lpzu, improvise-8jk6.13, improvise-8jk6.12)
ImportPipeline::from_merge seeds the File items and every file's columns from Merged::sources, so a header-only CSV keeps its File item and its columns (unseen columns are proposed as Labels, as an all-null column is). A file with no measure column gets a records view filtered to it instead of paging onto another file's hidden measure. Four per-file-view rules the review found untested now each have a test that fails when the rule is broken. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
This commit is contained in:
committed by
Edward Langley
parent
005b7a769a
commit
c43ee72402
@@ -691,8 +691,41 @@ so `Plan::place` gives it no coordinate there (improvise-8jk6.2 above), and no
|
||||
measure cell. Nothing is fabricated: no `0` and no `""`. The analyzer already
|
||||
classified each column over the values present (improvise-l1f5), so a column
|
||||
only one file has is judged by that file's values alone. The category cap
|
||||
counts the **union** of every file's categories. Rows from different files
|
||||
cannot collide, because `File` differs.
|
||||
counts the **union** of every file's categories, header-only files included
|
||||
(see the next paragraph). Rows from different files cannot collide, because
|
||||
`File` differs. One caveat on "judged by its values": a JSON `""` is not yet
|
||||
dropped by the analyzer the way `null` is, so a JSON numeric column holding
|
||||
`""` is proposed as a Label (improvise-8jk6.11). CSV is unaffected, because
|
||||
its parser turns a blank field into `null`.
|
||||
|
||||
**A header-only file is still a file** (the Codex P2 on PR #314). code-maat
|
||||
writes a header and no rows whenever nothing passes its threshold, so
|
||||
`coupling.csv` is often exactly that. The pipeline is analyzed from *records*,
|
||||
and such a file has none, so it had no `File` item, its columns never reached
|
||||
the model, and its view filtered to an item that did not exist.
|
||||
`ImportPipeline::from_merge` therefore **seeds from `Merged::sources`**:
|
||||
|
||||
- the `File` proposal's items are the sources' items, in file order, so every
|
||||
file has its item whether it has rows or not;
|
||||
- a source column that no record has a value in is proposed as a **Label**.
|
||||
That is the analyzer's own verdict for an all-null column (improvise-l1f5),
|
||||
and a column with no values is exactly that: nothing can be classified, so it
|
||||
takes no pivot slot and does not count toward the category cap until the user
|
||||
reclassifies it in the wizard or with `--category`/`--measure`.
|
||||
|
||||
This is also the file identity the user wants kept for a future update-import.
|
||||
|
||||
**A file with no measure column gets a records view** (improvise-8jk6.13). A
|
||||
record with no measure makes no cell, so such a file contributes items and no
|
||||
data. Its view had nothing to page `_Measure` to, and paged onto another file's
|
||||
measure, which the same view hides; the cell there was a `Record`, editable,
|
||||
under this file's `File`. Now `source_view` gives such a file the records
|
||||
layout (`view::records_steps`, the one home of what a records view is) with
|
||||
`File` still on Filter at the file's item. It draws zero rows, because there
|
||||
are no cells. A header-only file whose columns are
|
||||
all Labels is the same case. In records mode the columns are the whole model's
|
||||
categories and measures, not just this file's: records mode has no per-view
|
||||
column filter. With no rows that is cosmetic.
|
||||
|
||||
**Each file gets a view** (the user, 2026-09-25: "dir import should do this:
|
||||
one view filtered by the file column and projected to the columns in that
|
||||
@@ -721,7 +754,12 @@ initial view. Each per-file view:
|
||||
records lack those, so nothing splits.
|
||||
- **Puts `_Measure` on Page**, as `Workbook::new` does, **with other files'
|
||||
measures hidden** and the page selection on the file's first measure. A
|
||||
formula target comes from no file, so it stays visible.
|
||||
formula target comes from no file, so it stays visible. A file with no measure
|
||||
gets a records view instead (above).
|
||||
- **Is never named like another view from the same import.** `Default.csv` and
|
||||
`Default 2.csv` become `Default 2` and `Default 2 2`; without the same-batch
|
||||
check both would be named `Default 2` and one view would silently replace the
|
||||
other.
|
||||
|
||||
A per-file view is made only while `File` is an accepted category. A user who
|
||||
skipped or reclassified it has no filter to build views from.
|
||||
|
||||
@@ -444,7 +444,9 @@ or browser test was written. For each file's view:
|
||||
- the drawn cells are that file's alone: `2.00, 7.00` for age and
|
||||
`5.00, 3.00` for coupling.
|
||||
|
||||
Rivals R2, R7 and R9 each fail it.
|
||||
Rivals R2, R7 and R9 each fail it. Since the review of `4e68372c` the fixture
|
||||
also holds a header-only `soc.csv`. Its view must carry `File` at `soc`, be a
|
||||
records view, and draw zero rows. Rivals F7 and F8 fail that half.
|
||||
|
||||
### a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not
|
||||
|
||||
|
||||
@@ -1702,6 +1702,14 @@ improvise-lpzu. `[z.csv, b/, a.csv]` must expand to
|
||||
is sorted within itself. A fixture listing only folders, or files in name
|
||||
order, could not tell "argument order" from "sort everything". R6 fails it.
|
||||
|
||||
### two_files_with_one_stem_union_their_columns
|
||||
|
||||
improvise-8jk6.12 (3). `a/x.csv` (`entity,one`) and `b/x.csv` (`entity,two`) are
|
||||
one `File` item, `x`, whose columns are `entity, one, two`. Rival F3 (the
|
||||
reviewer's fault: clear the stem's columns before each file) leaves
|
||||
`entity, two`, and the `x` view would then hide the measure `one` that half its
|
||||
records carry.
|
||||
|
||||
### files_with_different_headers_merge_without_inventing_the_missing_columns
|
||||
|
||||
improvise-lpzu, at the parser. Each merged record carries exactly its own
|
||||
@@ -1767,3 +1775,52 @@ identical `.improv` text, and `format(parse(text)) == text`. The reloaded
|
||||
document still has the three views in order, `File: filter` at the file's item,
|
||||
and the same drawn values. This is the first persisted, non-virtual use of
|
||||
`Axis::Filter` (gotcha 27). R7 and R9 fail it.
|
||||
|
||||
### a_header_only_file_keeps_its_file_item_its_columns_and_its_view
|
||||
|
||||
The ChatGPT Codex P2 on PR #314 at `4e68372c`, confirmed by the adversarial
|
||||
review. `age.csv` (one row), a header-only `coupling.csv` and `rev.csv` (one
|
||||
row). At `005b7a76` the test failed at the first assertion: `File` held only
|
||||
`age` and `rev`, so the `coupling` view filtered to an item that did not exist,
|
||||
and `coupled`, `degree` and `average-revs` were not in the model. Now `File`
|
||||
reads `age, coupling, rev` in file order, the three columns exist (as Labels),
|
||||
and the `coupling` view is a zero-row records view filtered to `coupling`, while
|
||||
the `age` view still draws `2`. Rival F5 (sources not seeded) and F6 (unseen
|
||||
columns not proposed) each fail it, and so do F7 and F8 below.
|
||||
|
||||
### a_file_with_no_measure_gets_a_records_view_not_another_files_measure
|
||||
|
||||
improvise-8jk6.13, from the review of PR #314. `a.csv` (`entity,x`) and
|
||||
`tags.csv` (`entity,tag`). At `005b7a76` the `tags` view had `_Measure` on Page
|
||||
with no selection, so the layout paged onto `x`, a measure the same view hides.
|
||||
The test failed there, with `row_count() == 1`: `a`'s record, drawn under
|
||||
`File=tags`'s view. Now the view has no `_Measure` page, is a records layout,
|
||||
and draws zero rows, while `a`'s view is still a pivot. Rival F7 (the old
|
||||
paging) fails it, and F8 (the File selection lost when `records_steps` moves
|
||||
`File` off Filter) fails it with the same `1`. F8 is the mistake the first
|
||||
version of the fix made.
|
||||
|
||||
### a_files_view_seats_the_date_parts_of_its_own_date_column
|
||||
|
||||
improvise-8jk6.12 (1). The reviewer replaced the date-parts clause of
|
||||
`source_view`'s `own` with `false` and every test passed, because no folder
|
||||
fixture extracted a date part. Here `abs-churn.csv` has a `date` column,
|
||||
extracted to `date_Month`, and `age.csv` does not. `date_Month` is seated in the
|
||||
`abs-churn` view and on None in the `age` view. Rival F1 (the reviewer's fault)
|
||||
puts it on None in both and fails the first assertion.
|
||||
|
||||
### two_files_whose_names_collide_in_one_batch_keep_a_view_each
|
||||
|
||||
improvise-8jk6.12 (2). `Default.csv` and `Default 2.csv`. The first takes
|
||||
`Default 2`, so the second must take `Default 2 2`. Without the same-batch
|
||||
check both are named `Default 2`, and `IndexMap::insert` silently replaces the
|
||||
first, leaving two views where three are owed. The test asserts all three names
|
||||
with their filters. Rival F2 (the reviewer's fault) fails it.
|
||||
|
||||
### a_whitespace_only_category_value_is_no_coordinate
|
||||
|
||||
improvise-8jk6.12 (4). A `region` of `" "` is kept as a record with no
|
||||
`region` coordinate, and `region` holds only `East`. The trim in `scalar_name`
|
||||
predates this PR (improvise-ych), but the sparse-record rule now rests on it:
|
||||
without it the value becomes an item named `" "`. Rival F4 (the reviewer's
|
||||
fault) fails it.
|
||||
|
||||
Reference in New Issue
Block a user