Merge commit 'c91cc055c8ad81afd8f055b71c74af6819020637' into fix/headless-import-refusals

# Conflicts:
#	context/test-evidence/cli.md
#	context/test-evidence/formula-and-io.md
#	crates/improvise-io/src/import/wizard.rs
#	src/main.rs
#	tests/folder_import_views.rs
#	tests/headless_cli.rs
This commit is contained in:
Claude (agent-headless-import)
2026-09-24 22:08:36 -07:00
committed by Edward Langley
10 changed files with 374 additions and 28 deletions
+2 -2
View File
@@ -180,8 +180,8 @@ The import wizard analyzes CSV columns and proposes each as a category
year/month/quarter extraction), or skip. Multiple CSVs merge automatically year/month/quarter extraction), or skip. Multiple CSVs merge automatically
with a synthetic "File" category, even when their headers differ. A record with a synthetic "File" category, even when their headers differ. A record
simply has no value for a column its file lacks. A folder argument imports simply has no value for a column its file lacks. A folder argument imports
every `*.csv` in it, in name order, and each merged file also gets its own view, every `*.csv` in it, in name order, even when it holds a single CSV. Each file
filtered to that file and showing only its columns: also gets its own view, filtered to that file and showing only its columns:
```sh ```sh
improvise import reports/ --no-wizard -o reports.improv improvise import reports/ --no-wizard -o reports.improv
+3 -1
View File
@@ -373,7 +373,9 @@ there is deliberately no flag to name the category yet.
## 12 — `import` takes folders, and the CLI only says which arguments are folders (improvise-lpzu) ## 12 — `import` takes folders, and the CLI only says which arguments are folders (improvise-lpzu)
`improvise import` takes any mix of files and folders. A folder stands for its `improvise import` takes any mix of files and folders. A folder stands for its
`*.csv` files, in name order, so `import dir/` is `import dir/*.csv`. The rules `*.csv` files, in name order. A folder always merges, even when it holds a
single CSV, so that file keeps its `File` item and view. A single file named
directly still imports alone (the user's ruling on the PR). The rules
and their reasons are in `context/code/formula-and-io.md`, "A folder of CSVs is and their reasons are in `context/code/formula-and-io.md`, "A folder of CSVs is
one sparse model, with a view per file". The CLI's part is the one question one sparse model, with a view per file". The CLI's part is the one question
`Storage` cannot answer: whether a path is a folder. `get_import_data` asks `Storage` cannot answer: whether a path is a folder. `get_import_data` asks
+68 -10
View File
@@ -734,10 +734,28 @@ columns not in a given CSV file." So files with different headers merge into
takes the arguments in order and replaces each folder with its `*.csv` takes the arguments in order and replaces each folder with its `*.csv`
entries. The extension check is case-insensitive, the listing is not recursive, entries. The extension check is case-insensitive, the listing is not recursive,
and the order is `Storage::list`'s, which is sorted by contract. Then the and the order is `Storage::list`'s, which is sorted by contract. Then the
existing path runs unchanged: two or more files merge through `merge_csvs_from` existing path runs, with one difference: **a folder always merges**. Two or more
with a `File` category, and one file imports alone. So `import dir/` is files merge through `merge_csvs_from` with a `File` category, and so does a
`import dir/*.csv`, including when the folder holds a single CSV. Four cases folder holding a single CSV. That gives the one file its `File` item and its
follow from that equivalence rather than being chosen one by one: per-file view.
The single-CSV rule is **the user's ruling (2026-09-25)**, which replaced this
PR's first answer, "`import dir/` is exactly `import dir/*.csv`". Their words:
"yes, we might want to have some kind of update import eventually (don't plan
this yet, but we'd want file identity for this)". A folder names a collection
that may grow, so a document built from it should record which file each
record came from from the start.
**A single CSV named directly, not through a folder, still imports alone,
without `File`.** That is deliberate. Naming one file asks for that file, not
for a collection, and the shipped recipes (`examples/demo.improv` from
`examples/demo.csv`) and every existing single-file script would otherwise gain
a `File` category and a second view. File identity for update-import can be
asked for by naming a folder. The test is
`a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not`.
Apart from that, a folder still behaves as the glob would, and four cases follow
from that rather than being chosen one by one:
- **Non-CSV entries are skipped**, not refused, because a glob skips them. This - **Non-CSV entries are skipped**, not refused, because a glob skips them. This
includes the output `.improv`, which a second run finds in the folder. includes the output `.improv`, which a second run finds in the folder.
@@ -761,8 +779,41 @@ so `Plan::place` gives it no coordinate there (improvise-8jk6.2 above), and no
measure cell. Nothing is fabricated: no `0` and no `""`. The analyzer already measure cell. Nothing is fabricated: no `0` and no `""`. The analyzer already
classified each column over the values present (improvise-l1f5), so a column classified each column over the values present (improvise-l1f5), so a column
only one file has is judged by that file's values alone. The category cap only one file has is judged by that file's values alone. The category cap
counts the **union** of every file's categories. Rows from different files counts the **union** of every file's categories, header-only files included
cannot collide, because `File` differs. (see the next paragraph). Rows from different files cannot collide, because
`File` differs. One caveat on "judged by its values": a JSON `""` is not yet
dropped by the analyzer the way `null` is, so a JSON numeric column holding
`""` is proposed as a Label (improvise-8jk6.11). CSV is unaffected, because
its parser turns a blank field into `null`.
**A header-only file is still a file** (the Codex P2 on PR #314). code-maat
writes a header and no rows whenever nothing passes its threshold, so
`coupling.csv` is often exactly that. The pipeline is analyzed from *records*,
and such a file has none, so it had no `File` item, its columns never reached
the model, and its view filtered to an item that did not exist.
`ImportPipeline::from_merge` therefore **seeds from `Merged::sources`**:
- the `File` proposal's items are the sources' items, in file order, so every
file has its item whether it has rows or not;
- a source column that no record has a value in is proposed as a **Label**.
That is the analyzer's own verdict for an all-null column (improvise-l1f5),
and a column with no values is exactly that: nothing can be classified, so it
takes no pivot slot and does not count toward the category cap until the user
reclassifies it in the wizard or with `--category`/`--measure`.
This is also the file identity the user wants kept for a future update-import.
**A file with no measure column gets a records view** (improvise-8jk6.13). A
record with no measure makes no cell, so such a file contributes items and no
data. Its view had nothing to page `_Measure` to, and paged onto another file's
measure, which the same view hides; the cell there was a `Record`, editable,
under this file's `File`. Now `source_view` gives such a file the records
layout (`view::records_steps`, the one home of what a records view is) with
`File` still on Filter at the file's item. It draws zero rows, because there
are no cells. A header-only file whose columns are
all Labels is the same case. In records mode the columns are the whole model's
categories and measures, not just this file's: records mode has no per-view
column filter. With no rows that is cosmetic.
**Each file gets a view** (the user, 2026-09-25: "dir import should do this: **Each file gets a view** (the user, 2026-09-25: "dir import should do this:
one view filtered by the file column and projected to the columns in that one view filtered by the file column and projected to the columns in that
@@ -791,7 +842,12 @@ initial view. Each per-file view:
records lack those, so nothing splits. records lack those, so nothing splits.
- **Puts `_Measure` on Page**, as `Workbook::new` does, **with other files' - **Puts `_Measure` on Page**, as `Workbook::new` does, **with other files'
measures hidden** and the page selection on the file's first measure. A measures hidden** and the page selection on the file's first measure. A
formula target comes from no file, so it stays visible. formula target comes from no file, so it stays visible. A file with no measure
gets a records view instead (above).
- **Is never named like another view from the same import.** `Default.csv` and
`Default 2.csv` become `Default 2` and `Default 2 2`; without the same-batch
check both would be named `Default 2` and one view would silently replace the
other.
A per-file view is made only while `File` is an accepted category. A user who A per-file view is made only while `File` is an accepted category. A user who
skipped or reclassified it has no filter to build views from. skipped or reclassified it has no filter to build views from.
@@ -852,9 +908,11 @@ test is `a_record_with_no_coordinate_is_drawn_only_where_that_category_is_off_th
- **`.improv`:** its Data line omits the category, and it round-trips. - **`.improv`:** its Data line omits the category, and it round-trips.
- **Drill:** a drill that does not constrain the category reaches it. - **Drill:** a drill that does not constrain the category reaches it.
Whether Row/Column/Page should give such records a place of their own (a **No "(none)" position, by the user's decision (2026-09-25).** They were asked
"(none)" position) is **open, and it is the user's call**. It is raised on the whether Row, Column or Page should give such records a place of their own. Their
lpzu PR. Nothing here invents one. answer: "I think leaving it out for now is fine, maybe a UI element somewhere
that shows the number of records underneath the current view?" That indicator
is improvise-gtn7.10, and it is not an import concern.
### Colliding records are a question, not a default (improvise-cgqt) ### Colliding records are a question, not a default (improvise-cgqt)
+14 -1
View File
@@ -444,7 +444,9 @@ or browser test was written. For each file's view:
- the drawn cells are that file's alone: `2.00, 7.00` for age and - the drawn cells are that file's alone: `2.00, 7.00` for age and
`5.00, 3.00` for coupling. `5.00, 3.00` for coupling.
Rivals R2, R7 and R9 each fail it. Rivals R2, R7 and R9 each fail it. Since the review of `4e68372c` the fixture
also holds a header-only `soc.csv`. Its view must carry `File` at `soc`, be a
records view, and draw zero rows. Rivals F7 and F8 fail that half.
### a_csv_column_forced_to_category_keeps_its_text_verbatim ### a_csv_column_forced_to_category_keeps_its_text_verbatim
@@ -494,3 +496,14 @@ decision and not a drift. It asserts the warning's count (`1 of 3`), exit 0,
and a Data line from the record *after* the skipped one. That last check and a Data line from the record *after* the skipped one. That last check
shows the skip did not end the import early. **R9**, making any skip fatal, shows the skip did not end the import early. **R9**, making any skip fatal,
fails it. fails it.
### a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not
improvise-lpzu, after the user's ruling on PR #314 ("we'd want file identity").
At `4e68372c`, a folder holding one CSV took the single-file path, because
`import dir/` was exactly `dir/*.csv`. That document had no `File` category and
no per-file view, and the test failed at the `## Category: File` assertion. The
other half pins the direction the ruling did *not* change: the same CSV named
directly imports without any `File`. The rival that merges every all-CSV
import, lone files included, fails that half ("a lone file keeps today's
shape"). So the test separates "a folder merges" from "everything merges".
+57
View File
@@ -1712,6 +1712,14 @@ improvise-lpzu. `[z.csv, b/, a.csv]` must expand to
is sorted within itself. A fixture listing only folders, or files in name is sorted within itself. A fixture listing only folders, or files in name
order, could not tell "argument order" from "sort everything". R6 fails it. order, could not tell "argument order" from "sort everything". R6 fails it.
### two_files_with_one_stem_union_their_columns
improvise-8jk6.12 (3). `a/x.csv` (`entity,one`) and `b/x.csv` (`entity,two`) are
one `File` item, `x`, whose columns are `entity, one, two`. Rival F3 (the
reviewer's fault: clear the stem's columns before each file) leaves
`entity, two`, and the `x` view would then hide the measure `one` that half its
records carry.
### files_with_different_headers_merge_without_inventing_the_missing_columns ### files_with_different_headers_merge_without_inventing_the_missing_columns
improvise-lpzu, at the parser. Each merged record carries exactly its own improvise-lpzu, at the parser. Each merged record carries exactly its own
@@ -1837,3 +1845,52 @@ where the user can still choose another array path). On the base the step
refused with `No categories: … Press c to cycle a field's kind`, which offers refused with `No categories: … Press c to cycle a field's kind`, which offers
a key that has no field to act on. **R8**, the step without the check, fails a key that has no field to act on. **R8**, the step without the check, fails
it. The build test does not see R8. it. The build test does not see R8.
### a_header_only_file_keeps_its_file_item_its_columns_and_its_view
The ChatGPT Codex P2 on PR #314 at `4e68372c`, confirmed by the adversarial
review. `age.csv` (one row), a header-only `coupling.csv` and `rev.csv` (one
row). At `005b7a76` the test failed at the first assertion: `File` held only
`age` and `rev`, so the `coupling` view filtered to an item that did not exist,
and `coupled`, `degree` and `average-revs` were not in the model. Now `File`
reads `age, coupling, rev` in file order, the three columns exist (as Labels),
and the `coupling` view is a zero-row records view filtered to `coupling`, while
the `age` view still draws `2`. Rival F5 (sources not seeded) and F6 (unseen
columns not proposed) each fail it, and so do F7 and F8 below.
### a_file_with_no_measure_gets_a_records_view_not_another_files_measure
improvise-8jk6.13, from the review of PR #314. `a.csv` (`entity,x`) and
`tags.csv` (`entity,tag`). At `005b7a76` the `tags` view had `_Measure` on Page
with no selection, so the layout paged onto `x`, a measure the same view hides.
The test failed there, with `row_count() == 1`: `a`'s record, drawn under
`File=tags`'s view. Now the view has no `_Measure` page, is a records layout,
and draws zero rows, while `a`'s view is still a pivot. Rival F7 (the old
paging) fails it, and F8 (the File selection lost when `records_steps` moves
`File` off Filter) fails it with the same `1`. F8 is the mistake the first
version of the fix made.
### a_files_view_seats_the_date_parts_of_its_own_date_column
improvise-8jk6.12 (1). The reviewer replaced the date-parts clause of
`source_view`'s `own` with `false` and every test passed, because no folder
fixture extracted a date part. Here `abs-churn.csv` has a `date` column,
extracted to `date_Month`, and `age.csv` does not. `date_Month` is seated in the
`abs-churn` view and on None in the `age` view. Rival F1 (the reviewer's fault)
puts it on None in both and fails the first assertion.
### two_files_whose_names_collide_in_one_batch_keep_a_view_each
improvise-8jk6.12 (2). `Default.csv` and `Default 2.csv`. The first takes
`Default 2`, so the second must take `Default 2 2`. Without the same-batch
check both are named `Default 2`, and `IndexMap::insert` silently replaces the
first, leaving two views where three are owed. The test asserts all three names
with their filters. Rival F2 (the reviewer's fault) fails it.
### a_whitespace_only_category_value_is_no_coordinate
improvise-8jk6.12 (4). A `region` of `" "` is kept as a record with no
`region` coordinate, and `region` holds only `East`. The trim in `scalar_name`
predates this PR (improvise-ych), but the sparse-record rule now rests on it:
without it the value becomes an item named `" "`. Rival F4 (the reviewer's
fault) fails it.
@@ -524,6 +524,22 @@ mod tests {
); );
} }
/// improvise-8jk6.12 (3): two files with one stem are one item, and its
/// columns are both files'. Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn two_files_with_one_stem_union_their_columns() {
let fs = folder(&[
("a/x.csv", "entity,one\ne,1\n"),
("b/x.csv", "entity,two\ne,2\n"),
]);
let merged = merge_csvs_from(&fs, &["a/x.csv", "b/x.csv"]).unwrap();
let expected = SourceFile {
item: "x".to_string(),
columns: ["entity", "one", "two"].map(String::from).to_vec(),
};
assert_eq!(merged.sources, [expected]);
}
// ── RFC 4180 edge cases ─────────────────────────────────────────── // ── RFC 4180 edge cases ───────────────────────────────────────────
#[test] #[test]
+166 -7
View File
@@ -7,11 +7,11 @@ use super::analyzer::{
DateComponent, FieldKind, FieldProposal, analyze_records, extract_array_at_path, DateComponent, FieldKind, FieldProposal, analyze_records, extract_array_at_path,
extract_date_component, find_array_paths, extract_date_component, find_array_paths,
}; };
use super::csv_parser::{self, CsvRecords, FILE_CATEGORY, SourceFile}; use super::csv_parser::{self, CsvRecords, FILE_CATEGORY, Merged, SourceFile};
use crate::formula::{Formula, parse_formula}; use crate::formula::{Formula, parse_formula};
use crate::model::MAX_CATEGORIES; use crate::model::MAX_CATEGORIES;
use crate::model::cell::{CellKey, CellValue}; use crate::model::cell::{CellKey, CellValue};
use crate::view::{Axis, VIRTUAL_VIEW_NAMES, View}; use crate::view::{Axis, VIRTUAL_VIEW_NAMES, View, ViewStep, records_steps};
use crate::workbook::Workbook; use crate::workbook::Workbook;
// ── Pipeline (no UI state) ──────────────────────────────────────────────────── // ── Pipeline (no UI state) ────────────────────────────────────────────────────
@@ -129,6 +129,32 @@ impl ImportPipeline {
pipeline pipeline
} }
/// A pipeline over a CSV merge, carrying its sources for the per-file views.
pub fn from_merge(merged: Merged) -> Self {
let mut pipeline = Self::from_csv(merged.records);
pipeline.seed_sources(merged.sources);
pipeline
}
/// Every file keeps its `File` item and its columns, rows or none: a
/// header-only file is still a file (improvise-lpzu).
fn seed_sources(&mut self, sources: Vec<SourceFile>) {
let items: Vec<String> = sources.iter().map(|s| s.item.clone()).collect();
match self.proposals.iter_mut().find(|p| p.field == FILE_CATEGORY) {
Some(file) => file.distinct_values = items,
None => self
.proposals
.push(unseen(FILE_CATEGORY, FieldKind::Category, items)),
}
for column in sources.iter().flat_map(|s| &s.columns) {
if !self.proposals.iter().any(|p| &p.field == column) {
self.proposals
.push(unseen(column, FieldKind::Label, vec![]));
}
}
self.sources = sources;
}
pub fn select_path(&mut self, path: &str) { pub fn select_path(&mut self, path: &str) {
self.selected_path = path.to_string(); self.selected_path = path.to_string();
if let Some(arr) = extract_array_at_path(&self.raw, path) { if let Some(arr) = extract_array_at_path(&self.raw, path) {
@@ -436,6 +462,19 @@ impl ImportPipeline {
} }
} }
/// A proposal for a column no record has a value in, which is what an
/// all-null column already gets from the analyzer (improvise-l1f5).
fn unseen(field: &str, kind: FieldKind, distinct_values: Vec<String>) -> FieldProposal {
FieldProposal {
field: field.to_string(),
kind,
distinct_values,
accepted: true,
date_format: None,
date_components: vec![],
}
}
/// One view per merged file, in file order, while `File` is a category. The /// One view per merged file, in file order, while `File` is a category. The
/// shape and why Filter: `context/code/formula-and-io.md` (improvise-lpzu). /// shape and why Filter: `context/code/formula-and-io.md` (improvise-lpzu).
fn source_views(wb: &Workbook, sources: &[SourceFile], plan: &Plan) -> Vec<View> { fn source_views(wb: &Workbook, sources: &[SourceFile], plan: &Plan) -> Vec<View> {
@@ -484,12 +523,26 @@ fn source_view(wb: &Workbook, source: &SourceFile, plan: &Plan, name: String) ->
for measure in theirs { for measure in theirs {
view.hide_item("_Measure", &measure.field); view.hide_item("_Measure", &measure.field);
} }
if let Some(first) = mine.first() { match mine.first() {
view.set_page_selection("_Measure", &first.field); Some(first) => view.set_page_selection("_Measure", &first.field),
None => into_records(&mut view, wb, &source.item),
} }
view view
} }
/// A file with no measure has no measure to page to: its view is its records,
/// still filtered to the file (improvise-8jk6.13).
fn into_records(view: &mut View, wb: &Workbook, item: &str) {
for step in records_steps(&wb.model) {
match step {
ViewStep::Axis { category, axis } => view.set_axis(&category, axis),
ViewStep::PageSelection { category, item } => view.set_page_selection(&category, &item),
}
}
view.set_axis(FILE_CATEGORY, Axis::Filter);
view.set_page_selection(FILE_CATEGORY, item);
}
/// The accepted fields by role, and the date parts derived from them. /// The accepted fields by role, and the date parts derived from them.
struct Plan<'a> { struct Plan<'a> {
categories: Vec<&'a FieldProposal>, categories: Vec<&'a FieldProposal>,
@@ -2280,9 +2333,7 @@ mod tests {
let paths: Vec<&str> = files.iter().map(|(p, _)| *p).collect(); let paths: Vec<&str> = files.iter().map(|(p, _)| *p).collect();
let merged = merge_csvs_from(&fs, &paths).unwrap(); let merged = merge_csvs_from(&fs, &paths).unwrap();
assert!(merged.warnings.is_empty(), "{:?}", merged.warnings); assert!(merged.warnings.is_empty(), "{:?}", merged.warnings);
let mut p = ImportPipeline::from_csv(merged.records); ImportPipeline::from_merge(merged)
p.sources = merged.sources;
p
} }
/// code-maat's age and coupling reports. /// code-maat's age and coupling reports.
@@ -2370,6 +2421,114 @@ mod tests {
assert_eq!((drill("Payee", "Bolt"), drill("Date", "d1")), (1, 1)); assert_eq!((drill("Payee", "Bolt"), drill("Date", "d1")), (1, 1));
} }
/// improvise-lpzu, the Codex P2 on PR #314: a header-only file keeps its
/// identity. Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn a_header_only_file_keeps_its_file_item_its_columns_and_its_view() {
use crate::view::GridLayout;
let (wb, _) = merged_pipeline(&[
("maat/age.csv", "entity,age-months\nsrc/a.rs,2\n"),
("maat/coupling.csv", "entity,coupled,degree,average-revs\n"),
("maat/rev.csv", "entity,n-revs\nsrc/a.rs,9\n"),
])
.build_model_with_warnings()
.unwrap();
let files: Vec<&String> = wb.model.category("File").unwrap().items.keys().collect();
assert_eq!(files, ["age", "coupling", "rev"]);
for column in ["coupled", "degree", "average-revs"] {
assert!(wb.model.category(column).is_some(), "{column} is missing");
}
let view = &wb.views["coupling"];
assert_eq!(view.page_selection("File"), Some("coupling"));
let layout = GridLayout::with_frozen_records(&wb.model, view, None);
assert!(layout.is_records_mode() && layout.row_count() == 0);
assert_eq!(drawn(&wb, "age"), [2.0]);
}
/// improvise-8jk6.13: a view never pages onto a measure it hides.
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn a_file_with_no_measure_gets_a_records_view_not_another_files_measure() {
use crate::view::GridLayout;
let (wb, _) = merged_pipeline(&[
("t/a.csv", "entity,x\ne1,1\n"),
("t/tags.csv", "entity,tag\ne1,red\n"),
])
.build_model_with_warnings()
.unwrap();
let tags = &wb.views["tags"];
assert_eq!(tags.page_selection("_Measure"), None);
assert_ne!(tags.axis_of("_Measure"), Axis::Page, "{tags:?}");
let layout = GridLayout::with_frozen_records(&wb.model, tags, None);
assert!(layout.is_records_mode(), "{tags:?}");
assert_eq!(
layout.row_count(),
0,
"a record with no measure has no cell"
);
assert!(!wb.views["a"].is_records_layout());
}
/// improvise-8jk6.12 (1): a date part derived from the file's own column
/// is the file's. Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn a_files_view_seats_the_date_parts_of_its_own_date_column() {
use crate::import::analyzer::DateComponent;
let mut p = merged_pipeline(&[
("m/abs-churn.csv", "date,added\n2026-01-02,10\n"),
("m/age.csv", "entity,age-months\nsrc/a.rs,2\n"),
]);
let date = p.proposals.iter_mut().find(|f| f.field == "date").unwrap();
assert_eq!(date.kind, FieldKind::TimeCategory);
date.date_components = vec![DateComponent::Month];
let (wb, _) = p.build_model_with_warnings().unwrap();
assert_ne!(wb.views["abs-churn"].axis_of("date_Month"), Axis::None);
assert_eq!(wb.views["age"].axis_of("date_Month"), Axis::None);
}
/// improvise-8jk6.12 (2): two files of one batch never share a view name.
/// Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn two_files_whose_names_collide_in_one_batch_keep_a_view_each() {
let (wb, _) = merged_pipeline(&[
("d/Default.csv", "Region,Revenue\nEast,1\n"),
("d/Default 2.csv", "Region,Revenue\nEast,2\n"),
])
.build_model_with_warnings()
.unwrap();
let filters: Vec<(&str, Option<&str>)> = wb
.views
.values()
.map(|v| (v.name.as_str(), v.page_selection("File")))
.collect();
assert_eq!(
filters,
[
("Default", None),
("Default 2", Some("Default")),
("Default 2 2", Some("Default 2")),
]
);
}
/// improvise-8jk6.12 (4): a whitespace-only value is no coordinate, never
/// an item. Evidence: `context/test-evidence/formula-and-io.md`.
#[test]
fn a_whitespace_only_category_value_is_no_coordinate() {
let (wb, _) = ImportPipeline::new(json!([
{"region": "East", "revenue": 1.0},
{"region": " ", "revenue": 2.0},
]))
.build_model_with_warnings()
.unwrap();
let items: Vec<&String> = wb.model.category("region").unwrap().items.keys().collect();
assert_eq!(items, ["East"]);
assert_eq!(
stored(&wb),
["_Measure=revenue, region=East = 1", "_Measure=revenue = 2"]
);
}
/// improvise-lpzu: each merged file gets a view of its own records and /// improvise-lpzu: each merged file gets a view of its own records and
/// columns. Evidence: `context/test-evidence/formula-and-io.md`. /// columns. Evidence: `context/test-evidence/formula-and-io.md`.
#[test] #[test]
+6 -4
View File
@@ -348,8 +348,12 @@ fn get_import_data(args: &[PathBuf]) -> Option<ImportPipeline> {
}; };
let paths = paths.as_slice(); let paths = paths.as_slice();
let all_csv = paths.iter().all(|p| csv_path_p(p)); let all_csv = paths.iter().all(|p| csv_path_p(p));
// A folder always merges, so even one CSV keeps its file identity (lpzu).
let a_folder = classified
.iter()
.any(|a| matches!(a, ImportPath::Folder(_)));
if paths.len() > 1 { if paths.len() > 1 || a_folder {
if !all_csv { if !all_csv {
eprintln!("Multi-file import only supports CSV files"); eprintln!("Multi-file import only supports CSV files");
return None; return None;
@@ -360,9 +364,7 @@ fn get_import_data(args: &[PathBuf]) -> Option<ImportPipeline> {
for warning in &merged.warnings { for warning in &merged.warnings {
eprintln!("warning: {warning}"); eprintln!("warning: {warning}");
} }
let mut pipeline = ImportPipeline::from_csv(merged.records); Some(ImportPipeline::from_merge(merged))
pipeline.sources = merged.sources;
Some(pipeline)
} }
Err(e) => { Err(e) => {
eprintln!("CSV merge error: {e:#}"); eprintln!("CSV merge error: {e:#}");
+11 -3
View File
@@ -19,15 +19,16 @@ fn reloaded_folder_import() -> App {
"maat/coupling.csv", "maat/coupling.csv",
"entity,coupled,degree,average-revs\nsrc/a.rs,src/b.rs,80,5\nsrc/b.rs,src/c.rs,40,3\n", "entity,coupled,degree,average-revs\nsrc/a.rs,src/b.rs,80,5\nsrc/b.rs,src/c.rs,40,3\n",
), ),
("maat/soc.csv", "entity,soc\n"),
]; ];
for (path, body) in files { for (path, body) in files {
fs.write(std::path::Path::new(path), body.as_bytes()) fs.write(std::path::Path::new(path), body.as_bytes())
.unwrap(); .unwrap();
} }
let merged = merge_csvs_from(&fs, &files.map(|(p, _)| p)).unwrap(); let merged = merge_csvs_from(&fs, &files.map(|(p, _)| p)).unwrap();
let mut pipeline = ImportPipeline::from_csv(merged.records); let (wb, _) = ImportPipeline::from_merge(merged)
pipeline.sources = merged.sources; .build_model_with_warnings()
let (wb, _) = pipeline.build_model_with_warnings().unwrap(); .unwrap();
App::new(parse_md(&format_md(&wb)).unwrap(), None) App::new(parse_md(&format_md(&wb)).unwrap(), None)
} }
@@ -54,4 +55,11 @@ fn a_saved_per_file_view_projects_only_its_file() {
.collect(); .collect();
assert_eq!(drawn, texts, "{view}"); assert_eq!(drawn, texts, "{view}");
} }
// A header-only file: its item exists, and its view has nothing editable.
let soc = grid_in(&app, "soc", whole);
let file = soc.page_coords.iter().find(|p| p.cat == "File").unwrap();
assert_eq!(file.selected, "soc");
assert!(soc.records, "a file with no measure opens in records mode");
assert_eq!(soc.extent.rows, 0, "{soc:?}");
} }
+31
View File
@@ -822,3 +822,34 @@ fn an_import_that_skips_some_records_warns_with_a_count_and_still_writes() {
let doc = std::fs::read_to_string(&out_path).unwrap(); let doc = std::fs::read_to_string(&out_path).unwrap();
assert!(doc.contains("_Measure=x, a=m = 4"), "{doc}"); assert!(doc.contains("_Measure=x, a=m = 4"), "{doc}");
} }
/// improvise-lpzu, the user's ruling: a folder always merges, so even one CSV
/// keeps its file identity. Evidence: `context/test-evidence/cli.md`.
#[test]
fn a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not() {
let scratch = Scratch::new("folder_single");
let dir = scratch.path("one");
std::fs::create_dir_all(&dir).unwrap();
let csv = dir.join("age.csv");
std::fs::write(&csv, "entity,age-months\nsrc/a.rs,2\n").unwrap();
let from_folder = scratch.path("folder.improv");
let out = run_import(&dir, &from_folder, &[]);
assert!(out.status.success(), "{}", stderr_of(&out));
let doc = std::fs::read_to_string(&from_folder).unwrap();
assert!(doc.contains("## Category: File\n- age\n"), "{doc}");
assert!(doc.lines().any(|l| l == "File: filter, age"), "{doc}");
assert!(
doc.contains("File=age, _Measure=age-months, entity=|src/a.rs| = 2"),
"{doc}"
);
let from_file = scratch.path("file.improv");
let out = run_import(&csv, &from_file, &[]);
assert!(out.status.success(), "{}", stderr_of(&out));
let doc = std::fs::read_to_string(&from_file).unwrap();
assert!(
!doc.contains("File"),
"a lone file keeps today's shape: {doc}"
);
}