feat(import): a folder of one CSV keeps its File category and view (improvise-lpzu)
The user's ruling on PR #314: a folder always merges, so even a single CSV in it records its file identity. A CSV named directly still imports alone. The "(none)" question is answered as no position, per the user. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdfEGXA7aJ6ftMsCc9PJze
This commit is contained in:
committed by
Edward Langley
parent
4e68372c25
commit
005b7a769a
@@ -180,8 +180,8 @@ The import wizard analyzes CSV columns and proposes each as a category
|
|||||||
year/month/quarter extraction), or skip. Multiple CSVs merge automatically
|
year/month/quarter extraction), or skip. Multiple CSVs merge automatically
|
||||||
with a synthetic "File" category, even when their headers differ. A record
|
with a synthetic "File" category, even when their headers differ. A record
|
||||||
simply has no value for a column its file lacks. A folder argument imports
|
simply has no value for a column its file lacks. A folder argument imports
|
||||||
every `*.csv` in it, in name order, and each merged file also gets its own view,
|
every `*.csv` in it, in name order, even when it holds a single CSV. Each file
|
||||||
filtered to that file and showing only its columns:
|
also gets its own view, filtered to that file and showing only its columns:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
improvise import reports/ --no-wizard -o reports.improv
|
improvise import reports/ --no-wizard -o reports.improv
|
||||||
|
|||||||
+3
-1
@@ -373,7 +373,9 @@ there is deliberately no flag to name the category yet.
|
|||||||
## 12 — `import` takes folders, and the CLI only says which arguments are folders (improvise-lpzu)
|
## 12 — `import` takes folders, and the CLI only says which arguments are folders (improvise-lpzu)
|
||||||
|
|
||||||
`improvise import` takes any mix of files and folders. A folder stands for its
|
`improvise import` takes any mix of files and folders. A folder stands for its
|
||||||
`*.csv` files, in name order, so `import dir/` is `import dir/*.csv`. The rules
|
`*.csv` files, in name order. A folder always merges, even when it holds a
|
||||||
|
single CSV, so that file keeps its `File` item and view. A single file named
|
||||||
|
directly still imports alone (the user's ruling on the PR). The rules
|
||||||
and their reasons are in `context/code/formula-and-io.md`, "A folder of CSVs is
|
and their reasons are in `context/code/formula-and-io.md`, "A folder of CSVs is
|
||||||
one sparse model, with a view per file". The CLI's part is the one question
|
one sparse model, with a view per file". The CLI's part is the one question
|
||||||
`Storage` cannot answer: whether a path is a folder. `get_import_data` asks
|
`Storage` cannot answer: whether a path is a folder. `get_import_data` asks
|
||||||
|
|||||||
@@ -646,10 +646,28 @@ columns not in a given CSV file." So files with different headers merge into
|
|||||||
takes the arguments in order and replaces each folder with its `*.csv`
|
takes the arguments in order and replaces each folder with its `*.csv`
|
||||||
entries. The extension check is case-insensitive, the listing is not recursive,
|
entries. The extension check is case-insensitive, the listing is not recursive,
|
||||||
and the order is `Storage::list`'s, which is sorted by contract. Then the
|
and the order is `Storage::list`'s, which is sorted by contract. Then the
|
||||||
existing path runs unchanged: two or more files merge through `merge_csvs_from`
|
existing path runs, with one difference: **a folder always merges**. Two or more
|
||||||
with a `File` category, and one file imports alone. So `import dir/` is
|
files merge through `merge_csvs_from` with a `File` category, and so does a
|
||||||
`import dir/*.csv`, including when the folder holds a single CSV. Four cases
|
folder holding a single CSV. That gives the one file its `File` item and its
|
||||||
follow from that equivalence rather than being chosen one by one:
|
per-file view.
|
||||||
|
|
||||||
|
The single-CSV rule is **the user's ruling (2026-09-25)**, which replaced this
|
||||||
|
PR's first answer, "`import dir/` is exactly `import dir/*.csv`". Their words:
|
||||||
|
"yes, we might want to have some kind of update import eventually (don't plan
|
||||||
|
this yet, but we'd want file identity for this)". A folder names a collection
|
||||||
|
that may grow, so a document built from it should record which file each
|
||||||
|
record came from from the start.
|
||||||
|
|
||||||
|
**A single CSV named directly, not through a folder, still imports alone,
|
||||||
|
without `File`.** That is deliberate. Naming one file asks for that file, not
|
||||||
|
for a collection, and the shipped recipes (`examples/demo.improv` from
|
||||||
|
`examples/demo.csv`) and every existing single-file script would otherwise gain
|
||||||
|
a `File` category and a second view. File identity for update-import can be
|
||||||
|
asked for by naming a folder. The test is
|
||||||
|
`a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not`.
|
||||||
|
|
||||||
|
Apart from that, a folder still behaves as the glob would, and four cases follow
|
||||||
|
from that rather than being chosen one by one:
|
||||||
|
|
||||||
- **Non-CSV entries are skipped**, not refused, because a glob skips them. This
|
- **Non-CSV entries are skipped**, not refused, because a glob skips them. This
|
||||||
includes the output `.improv`, which a second run finds in the folder.
|
includes the output `.improv`, which a second run finds in the folder.
|
||||||
@@ -764,9 +782,11 @@ test is `a_record_with_no_coordinate_is_drawn_only_where_that_category_is_off_th
|
|||||||
- **`.improv`:** its Data line omits the category, and it round-trips.
|
- **`.improv`:** its Data line omits the category, and it round-trips.
|
||||||
- **Drill:** a drill that does not constrain the category reaches it.
|
- **Drill:** a drill that does not constrain the category reaches it.
|
||||||
|
|
||||||
Whether Row/Column/Page should give such records a place of their own (a
|
**No "(none)" position, by the user's decision (2026-09-25).** They were asked
|
||||||
"(none)" position) is **open, and it is the user's call**. It is raised on the
|
whether Row, Column or Page should give such records a place of their own. Their
|
||||||
lpzu PR. Nothing here invents one.
|
answer: "I think leaving it out for now is fine, maybe a UI element somewhere
|
||||||
|
that shows the number of records underneath the current view?" That indicator
|
||||||
|
is improvise-gtn7.10, and it is not an import concern.
|
||||||
|
|
||||||
### Colliding records are a question, not a default (improvise-cgqt)
|
### Colliding records are a question, not a default (improvise-cgqt)
|
||||||
|
|
||||||
|
|||||||
@@ -445,3 +445,14 @@ or browser test was written. For each file's view:
|
|||||||
`5.00, 3.00` for coupling.
|
`5.00, 3.00` for coupling.
|
||||||
|
|
||||||
Rivals R2, R7 and R9 each fail it.
|
Rivals R2, R7 and R9 each fail it.
|
||||||
|
|
||||||
|
### a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not
|
||||||
|
|
||||||
|
improvise-lpzu, after the user's ruling on PR #314 ("we'd want file identity").
|
||||||
|
At `4e68372c`, a folder holding one CSV took the single-file path, because
|
||||||
|
`import dir/` was exactly `dir/*.csv`. That document had no `File` category and
|
||||||
|
no per-file view, and the test failed at the `## Category: File` assertion. The
|
||||||
|
other half pins the direction the ruling did *not* change: the same CSV named
|
||||||
|
directly imports without any `File`. The rival that merges every all-CSV
|
||||||
|
import, lone files included, fails that half ("a lone file keeps today's
|
||||||
|
shape"). So the test separates "a folder merges" from "everything merges".
|
||||||
|
|||||||
+5
-1
@@ -348,8 +348,12 @@ fn get_import_data(args: &[PathBuf]) -> Option<ImportPipeline> {
|
|||||||
};
|
};
|
||||||
let paths = paths.as_slice();
|
let paths = paths.as_slice();
|
||||||
let all_csv = paths.iter().all(|p| csv_path_p(p));
|
let all_csv = paths.iter().all(|p| csv_path_p(p));
|
||||||
|
// A folder always merges, so even one CSV keeps its file identity (lpzu).
|
||||||
|
let a_folder = classified
|
||||||
|
.iter()
|
||||||
|
.any(|a| matches!(a, ImportPath::Folder(_)));
|
||||||
|
|
||||||
if paths.len() > 1 {
|
if paths.len() > 1 || a_folder {
|
||||||
if !all_csv {
|
if !all_csv {
|
||||||
eprintln!("Multi-file import only supports CSV files");
|
eprintln!("Multi-file import only supports CSV files");
|
||||||
return None;
|
return None;
|
||||||
|
|||||||
@@ -708,3 +708,34 @@ fn a_headless_import_takes_files_and_folders_together_and_refuses_an_empty_folde
|
|||||||
);
|
);
|
||||||
assert!(!scratch.path("never.improv").exists());
|
assert!(!scratch.path("never.improv").exists());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// improvise-lpzu, the user's ruling: a folder always merges, so even one CSV
|
||||||
|
/// keeps its file identity. Evidence: `context/test-evidence/cli.md`.
|
||||||
|
#[test]
|
||||||
|
fn a_folder_of_one_csv_still_gets_a_file_category_and_its_view_but_a_lone_file_does_not() {
|
||||||
|
let scratch = Scratch::new("folder_single");
|
||||||
|
let dir = scratch.path("one");
|
||||||
|
std::fs::create_dir_all(&dir).unwrap();
|
||||||
|
let csv = dir.join("age.csv");
|
||||||
|
std::fs::write(&csv, "entity,age-months\nsrc/a.rs,2\n").unwrap();
|
||||||
|
|
||||||
|
let from_folder = scratch.path("folder.improv");
|
||||||
|
let out = run_import(&dir, &from_folder, &[]);
|
||||||
|
assert!(out.status.success(), "{}", stderr_of(&out));
|
||||||
|
let doc = std::fs::read_to_string(&from_folder).unwrap();
|
||||||
|
assert!(doc.contains("## Category: File\n- age\n"), "{doc}");
|
||||||
|
assert!(doc.lines().any(|l| l == "File: filter, age"), "{doc}");
|
||||||
|
assert!(
|
||||||
|
doc.contains("File=age, _Measure=age-months, entity=|src/a.rs| = 2"),
|
||||||
|
"{doc}"
|
||||||
|
);
|
||||||
|
|
||||||
|
let from_file = scratch.path("file.improv");
|
||||||
|
let out = run_import(&csv, &from_file, &[]);
|
||||||
|
assert!(out.status.success(), "{}", stderr_of(&out));
|
||||||
|
let doc = std::fs::read_to_string(&from_file).unwrap();
|
||||||
|
assert!(
|
||||||
|
!doc.contains("File"),
|
||||||
|
"a lone file keeps today's shape: {doc}"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user