Two textual conflicts, and one the merge resolved cleanly and wrongly. effect.rs: #96 extracted the LayoutGeometry list into layout_geometry_cases() shared by two tests, one of which compares it against the wire vocabulary; my side had added ViewTarget arguments and carried the base's assert_eq!(cases.len(), 13). Kept #96's structure with my arguments and dropped the count -- the list-versus-vocabulary check is strictly stronger and a count beside it would be a second answer to the same question. Verified the count came from the merge base, not from my side, before dropping it. test-evidence/command.md: both sides appended entries at the same point. Kept both. app.rs auto-merged CLEANLY and did not compile: improvise-4dpj's new a_batch_that_leaves_records_settles_out_of_records_normal constructs effect::SetAxis, which now carries a view operand. Fixed with ViewTarget::Active -- the host's own key path, which is what that test is. Checked rather than assumed, per the brief: - Test names diffed BOTH ways against the union of the two parents, not the count. Union 1510, merged 1509, exactly one name absent and it is the intended one: a_hosts_none_cats_leak_into_another_views_formula_columns, the ac2z characterization test my own branch replaced with a_formulas_columns_survive_a_host_that_aggregates_differently. Nothing else lost, nothing unexpected gained. - #96's the_enumeration_names_every_variant_the_derive_declares is still discriminating against the reshaped ModelCommand variants: dropping TransposeAxes from every_model_command() fails it by name. - The test-support feature stays off in shipped builds: cargo tree --edges normal --target wasm32-unknown-unknown finds no serde_json. And the finding the brief asked for, which is the opposite of a disagreement: #96's switching_views_across_records_ness_settles_against_ the_view_switched_to is the HOST-side twin of the client fix in this branch. Same batch, same crossing in both directions, and independently the same phrase in its message. Neither is redundant -- nothing in the projection stream carries the host's ViewState, so a host-side guard cannot see a client's mode or the reverse. Cross-referenced in context/test-evidence/clients.md so neither is deleted as a duplicate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FC2jVn8Au7E9JPkw6bwXXH
This commit is contained in:
Edward Langley
@@ -21,4 +21,5 @@ unicode-width = "^0.2"
|
||||
web-time = "1"
|
||||
|
||||
[dev-dependencies]
|
||||
improvise-protocol = { path = "../improvise-protocol", features = ["test-support"] }
|
||||
tempfile = "3"
|
||||
|
||||
@@ -2652,6 +2652,90 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// The settle's `RecordsNormal` arm, driven from the mode it names. The
|
||||
/// `:` crossing above enters the settle in `Normal` both ways — the evidence.
|
||||
#[test]
|
||||
fn a_batch_that_leaves_records_settles_out_of_records_normal() {
|
||||
use crate::view::Axis;
|
||||
let mut wb = Workbook::new("T");
|
||||
wb.add_category("Region").unwrap();
|
||||
wb.add_item("Region", "East").unwrap();
|
||||
let mut app = App::new(wb, None);
|
||||
app.handle_key(KeyEvent::new(KeyCode::Char('R'), KeyModifiers::NONE))
|
||||
.unwrap();
|
||||
assert!(app.layout.is_records_mode(), "setup: the grid is records");
|
||||
assert_eq!(
|
||||
app.view_state.mode,
|
||||
AppMode::RecordsNormal,
|
||||
"setup: the mode handed to the settle must be RecordsNormal"
|
||||
);
|
||||
|
||||
app.apply_effects(vec![Box::new(super::super::effect::SetAxis {
|
||||
view: ViewTarget::Active,
|
||||
category: "_Index".into(),
|
||||
axis: Axis::None,
|
||||
})]);
|
||||
|
||||
assert!(
|
||||
!app.layout.is_records_mode(),
|
||||
"setup: the batch has to leave records, or nothing crossed"
|
||||
);
|
||||
assert_eq!(
|
||||
app.view_state.mode,
|
||||
AppMode::Normal,
|
||||
"a batch that left records stayed in RecordsNormal (improvise-4dpj)"
|
||||
);
|
||||
}
|
||||
|
||||
/// The batch `SwitchViewAtCursor` emits, both ways across records-ness:
|
||||
/// records-ness is auto-detected from the axes, so a view switch moves it.
|
||||
#[test]
|
||||
fn switching_views_across_records_ness_settles_against_the_view_switched_to() {
|
||||
for to_records in [false, true] {
|
||||
let mut wb = Workbook::new("T");
|
||||
wb.add_category("Region").unwrap();
|
||||
wb.add_item("Region", "East").unwrap();
|
||||
let mut app = App::new(wb, None);
|
||||
// `R` creates and enters `_Records`; the pivot view it left is the
|
||||
// other end of every crossing below.
|
||||
app.handle_key(KeyEvent::new(KeyCode::Char('R'), KeyModifiers::NONE))
|
||||
.unwrap();
|
||||
let records_view = app.view_state.active_view.clone();
|
||||
let (from, to) = match to_records {
|
||||
true => ("Default", records_view.as_str()),
|
||||
false => (records_view.as_str(), "Default"),
|
||||
};
|
||||
app.apply_effects(vec![Box::new(super::super::effect::SwitchView(
|
||||
from.into(),
|
||||
))]);
|
||||
assert_eq!(
|
||||
app.layout.is_records_mode(),
|
||||
!to_records,
|
||||
"setup: the switch has to CROSS, so it must start on the other side"
|
||||
);
|
||||
|
||||
app.apply_effects(vec![
|
||||
Box::new(super::super::effect::SwitchView(to.into())),
|
||||
super::super::effect::change_mode(AppMode::Normal),
|
||||
]);
|
||||
|
||||
assert_eq!(
|
||||
app.layout.is_records_mode(),
|
||||
to_records,
|
||||
"setup: the switch has to land, or nothing crossed"
|
||||
);
|
||||
let expected = match to_records {
|
||||
true => AppMode::RecordsNormal,
|
||||
false => AppMode::Normal,
|
||||
};
|
||||
assert_eq!(
|
||||
app.view_state.mode, expected,
|
||||
"the batch settled against the view it LEFT, not the one it \
|
||||
switched to (improvise-4dpj)"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// A `:` line that really runs settles through `ReturnToNormalUnlessMoved`,
|
||||
/// which an empty line never reaches. See the evidence.
|
||||
#[test]
|
||||
|
||||
@@ -1444,11 +1444,38 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// Effects declaring `LayoutGeometry` must move `document_config` — the
|
||||
/// production hash `model_state.dirty` is decided by (improvise-1xf4).
|
||||
/// The case list below is COMPLETE over the wire vocabulary, which the
|
||||
/// compiler maintains — improvise-26dm, `context/code/command.md`.
|
||||
#[test]
|
||||
fn layout_geometry_effects_actually_change_document_config() {
|
||||
let cases: Vec<Box<dyn Effect>> = vec![
|
||||
fn the_layout_geometry_cases_are_every_wire_effect_of_that_kind() {
|
||||
let app = grouped_app();
|
||||
let mut reachable: Vec<String> = crate::ui::reduce::tests::every_model_command()
|
||||
.iter()
|
||||
.flat_map(|cmd| crate::ui::reduce::model_effects(&app, cmd))
|
||||
.filter(|e| e.kind() == EffectKind::Model(ModelChange::LayoutGeometry))
|
||||
.map(|e| crate::ui::reduce::tests::effect_type_name(e.as_ref()))
|
||||
.collect();
|
||||
let mut listed: Vec<String> = layout_geometry_cases()
|
||||
.iter()
|
||||
.map(|e| crate::ui::reduce::tests::effect_type_name(e.as_ref()))
|
||||
.collect();
|
||||
reachable.sort_unstable();
|
||||
reachable.dedup();
|
||||
listed.sort_unstable();
|
||||
listed.dedup();
|
||||
assert_eq!(
|
||||
listed, reachable,
|
||||
"the LayoutGeometry case list and the wire vocabulary disagree. A name \
|
||||
only on the LEFT means every_model_command() has a hole, not that the \
|
||||
case is spurious; a name only on the RIGHT is an effect nothing here \
|
||||
exercises. Read `context/code/command.md` before deleting either side"
|
||||
);
|
||||
}
|
||||
|
||||
/// The `LayoutGeometry` effects, named once so two tests can ask the same
|
||||
/// list the same questions — improvise-26dm, `context/code/command.md`.
|
||||
fn layout_geometry_cases() -> Vec<Box<dyn Effect>> {
|
||||
vec![
|
||||
Box::new(AddCategory("Region".into())),
|
||||
Box::new(RemoveCategory("Month".into())),
|
||||
Box::new(CreateView("Second".into())),
|
||||
@@ -1482,10 +1509,14 @@ mod tests {
|
||||
Box::new(CycleAxis(ViewTarget::Active, "Type".into())),
|
||||
Box::new(SetNumberFormat(ViewTarget::Active, ",.3f".into())),
|
||||
Box::new(TogglePruneEmpty(ViewTarget::Active)),
|
||||
];
|
||||
// A COUNT, not a totality check: nothing forces a fourteenth here, and a
|
||||
// probe variant proved it — improvise-5ubs, `context/code/command.md`.
|
||||
assert_eq!(cases.len(), 13, "the LayoutGeometry effects were thirteen");
|
||||
]
|
||||
}
|
||||
|
||||
/// Effects declaring `LayoutGeometry` must move `document_config` — the
|
||||
/// production hash `model_state.dirty` is decided by (improvise-1xf4).
|
||||
#[test]
|
||||
fn layout_geometry_effects_actually_change_document_config() {
|
||||
let cases = layout_geometry_cases();
|
||||
for effect in cases {
|
||||
assert_eq!(
|
||||
effect.kind(),
|
||||
|
||||
@@ -308,7 +308,7 @@ pub fn reduce_key(app: &mut App, key: KeyEvent) -> Vec<Command> {
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
pub(crate) mod tests {
|
||||
use super::*;
|
||||
use crate::model::cell::CellValue;
|
||||
use crate::ui::app::AppMode;
|
||||
@@ -527,9 +527,9 @@ mod tests {
|
||||
app
|
||||
}
|
||||
|
||||
/// One value per `ModelCommand` variant. [`variant_name`] matches
|
||||
/// exhaustively, so a new variant fails the build until it is listed here.
|
||||
fn every_model_command() -> Vec<ModelCommand> {
|
||||
/// One value per `ModelCommand` variant — by hand, and NOT compiler-forced:
|
||||
/// [`variant_name`]'s exhaustive match only forces an arm there (PR #96).
|
||||
pub(crate) fn every_model_command() -> Vec<ModelCommand> {
|
||||
vec![
|
||||
ModelCommand::SetCell {
|
||||
cell: record_cell(&a_stored_cell()),
|
||||
@@ -708,7 +708,7 @@ mod tests {
|
||||
|
||||
/// An effect's type name, off the leading identifier of its derived `Debug`.
|
||||
/// `Effect` is a trait object, so `std::any::type_name` cannot reach it.
|
||||
fn effect_type_name(effect: &dyn Effect) -> String {
|
||||
pub(crate) fn effect_type_name(effect: &dyn Effect) -> String {
|
||||
format!("{effect:?}")
|
||||
.split(|c: char| !c.is_alphanumeric() && c != '_')
|
||||
.next()
|
||||
@@ -765,6 +765,25 @@ mod tests {
|
||||
assert_eq!(names.len(), total, "a variant is listed twice: {names:?}");
|
||||
}
|
||||
|
||||
/// ...and covers it WHOLLY, against serde's derive — the one enumeration the
|
||||
/// compiler maintains (improvise-5ubs). `context/code/command.md`.
|
||||
#[test]
|
||||
fn the_enumeration_names_every_variant_the_derive_declares() {
|
||||
let listed: Vec<String> = every_model_command()
|
||||
.iter()
|
||||
.map(|cmd| variant_name(cmd).to_string())
|
||||
.collect();
|
||||
let missed: Vec<String> = improvise_protocol::variants::declared::<ModelCommand>()
|
||||
.into_iter()
|
||||
.filter(|name| !listed.contains(name))
|
||||
.collect();
|
||||
assert!(
|
||||
missed.is_empty(),
|
||||
"every_model_command() never constructs {missed:?} — adding an arm to \
|
||||
variant_name is compiler-forced, adding a VALUE to the list is not"
|
||||
);
|
||||
}
|
||||
|
||||
/// A model `Command` applied authoritatively lands where the keypress bound
|
||||
/// to the same mutation lands, `Dirty` included.
|
||||
#[test]
|
||||
|
||||
@@ -10,14 +10,19 @@ pub fn format_value(v: Option<&CellValue>, comma: bool, decimals: u8) -> String
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse a number format string like ",.0" into (use_commas, decimal_places).
|
||||
/// The digits a precision is named by: everything up to the first non-digit.
|
||||
fn leading_digits(s: &str) -> &str {
|
||||
&s[..s.find(|c: char| !c.is_ascii_digit()).unwrap_or(s.len())]
|
||||
}
|
||||
|
||||
/// Parse a printf-style number format like ",.2f" into (use_commas, decimals).
|
||||
/// Which spellings are legal: `context/code/core.md`, "The number format".
|
||||
pub fn parse_number_format(fmt: &str) -> (bool, u8) {
|
||||
let comma = fmt.contains(',');
|
||||
let decimals = fmt
|
||||
.rfind('.')
|
||||
.and_then(|i| fmt[i + 1..].parse::<u8>().ok())
|
||||
.unwrap_or(0);
|
||||
(comma, decimals)
|
||||
let precision = fmt
|
||||
.rsplit_once('.')
|
||||
.map_or("", |(_, tail)| leading_digits(tail));
|
||||
(comma, precision.parse::<u8>().unwrap_or(0))
|
||||
}
|
||||
|
||||
/// Round half away from zero (the "normal" rounding people expect).
|
||||
@@ -100,6 +105,47 @@ mod tests {
|
||||
assert_eq!(parse_number_format(",.1.3"), (true, 3));
|
||||
}
|
||||
|
||||
/// Bug improvise-711n: a trailing conversion letter made the whole suffix
|
||||
/// unparseable. Evidence: `context/test-evidence/core.md`.
|
||||
#[test]
|
||||
fn a_trailing_conversion_letter_does_not_erase_the_precision() {
|
||||
assert_eq!(parse_number_format(",.2f"), (true, 2));
|
||||
assert_eq!(parse_number_format(".2f"), (false, 2));
|
||||
assert_eq!(parse_number_format("%.2f"), (false, 2));
|
||||
// The precision is the LEADING digit run, not the tail with its
|
||||
// non-digits trimmed off — these two readings differ here.
|
||||
assert_eq!(parse_number_format(".2f3"), (false, 2));
|
||||
}
|
||||
|
||||
/// One convention, not two: `.00` is printf's "precision zero", never
|
||||
/// Excel's "two places". Evidence: `context/test-evidence/core.md`.
|
||||
#[test]
|
||||
fn a_zero_padded_picture_string_is_read_as_printf_not_as_excel() {
|
||||
assert_eq!(parse_number_format("0.00"), (false, 0));
|
||||
assert_eq!(parse_number_format(",.0f"), (true, 0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_precision_too_large_to_name_reads_as_none() {
|
||||
assert_eq!(parse_number_format(".300"), (false, 0));
|
||||
assert_eq!(parse_number_format(",.999f"), (true, 0));
|
||||
}
|
||||
|
||||
/// `format:` is `rest_of_line` in the grammar, so the format string can hold
|
||||
/// any text at all — including a multibyte char right after the dot.
|
||||
#[test]
|
||||
fn a_multibyte_character_after_the_dot_does_not_split_a_char_boundary() {
|
||||
assert_eq!(parse_number_format(",.2\u{20ac}"), (true, 2));
|
||||
assert_eq!(parse_number_format(".\u{20ac}"), (false, 0));
|
||||
assert_eq!(parse_number_format(".\u{0662}"), (false, 0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_format_naming_no_precision_at_all_reads_as_zero() {
|
||||
assert_eq!(parse_number_format(",f"), (true, 0));
|
||||
assert_eq!(parse_number_format(".f"), (false, 0));
|
||||
}
|
||||
|
||||
// ── format_f64 basic ───────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
|
||||
+223
-18
@@ -1,7 +1,7 @@
|
||||
//! One fit, end to end: a [`View`]'s records in, an improvise [`Formula`] and the
|
||||
//! evidence for believing it out. It never prints — `context/code/gp.md` §14.
|
||||
|
||||
use gp_tree::data::{Forced, Overrides, Target, preprocess};
|
||||
use gp_tree::data::{Forced, Overrides, Target, parse_datetime, preprocess};
|
||||
use gp_tree::fit::{
|
||||
FitConfig, RegressionQuality, cv_regress, denormalize, fit, importance_ranking, mse,
|
||||
permutation_importance, regression_quality, standardize,
|
||||
@@ -13,6 +13,31 @@ use improvise_formula::Formula;
|
||||
|
||||
use crate::dataset::{FeatureTable, feature_table};
|
||||
|
||||
/// Which branch `preprocess` would take for this column. It mirrors that function's
|
||||
/// own `match`, arm for arm — the preflight owes every branch, `context/code/gp.md` §14.
|
||||
fn inferred_kind(present: &[&str], is_document_category: bool) -> Forced {
|
||||
match present.is_empty() {
|
||||
// `preprocess` SKIPS an entirely-missing column. Counting it as one
|
||||
// categorical over-counts by one, which is the safe direction.
|
||||
true => Forced::Categorical,
|
||||
false if is_document_category => Forced::Categorical,
|
||||
false if present.iter().all(|c| c.parse::<f64>().is_ok()) => Forced::Numeric,
|
||||
false if present.iter().all(|c| parse_datetime(c).is_some()) => Forced::Date,
|
||||
false => Forced::Categorical,
|
||||
}
|
||||
}
|
||||
|
||||
/// `build_date` emits five parts, or seven once any cell carries a time.
|
||||
fn date_parts(present: &[&str]) -> usize {
|
||||
match present
|
||||
.iter()
|
||||
.any(|c| parse_datetime(c).is_some_and(|d| d.has_time))
|
||||
{
|
||||
true => 7,
|
||||
false => 5,
|
||||
}
|
||||
}
|
||||
|
||||
/// A cheap upper bound on how wide `preprocess` would expand this table, computed
|
||||
/// without expanding it. **Advisory, not authoritative** — `context/code/gp.md` §14.
|
||||
fn encoded_width(table: &FeatureTable, category_columns: usize) -> usize {
|
||||
@@ -29,20 +54,21 @@ fn encoded_width(table: &FeatureTable, category_columns: usize) -> usize {
|
||||
.copied()
|
||||
.filter(|c| !gp_tree::data::is_missing(c))
|
||||
.collect();
|
||||
let has_missing = present.len() < cells.len();
|
||||
// A document category is forced categorical below, so the preflight
|
||||
// counts it by cardinality too. Where the two still differ: gp.md §14.
|
||||
let numeric = col >= category_columns
|
||||
&& !present.is_empty()
|
||||
&& present.iter().all(|c| c.parse::<f64>().is_ok());
|
||||
match numeric {
|
||||
// A numeric column is one feature, plus the `.missing` flag.
|
||||
true => 1 + usize::from(has_missing),
|
||||
// Everything else: one column per distinct value.
|
||||
false => present
|
||||
// Every branch adds the `.missing` flag on the same terms, so it is
|
||||
// added once here rather than in each arm.
|
||||
let missing = usize::from(present.len() < cells.len());
|
||||
// No wildcard: a new `Forced` variant must be counted here rather
|
||||
// than silently falling into the categorical arm.
|
||||
match inferred_kind(&present, col < category_columns) {
|
||||
Forced::Numeric => 1 + missing,
|
||||
Forced::Date => date_parts(&present) + missing,
|
||||
// One column per distinct value, plus the one `build_categorical`
|
||||
// reserves for `(missing)` — gp.md §14.
|
||||
Forced::Categorical => (present
|
||||
.iter()
|
||||
.collect::<std::collections::HashSet<_>>()
|
||||
.len()
|
||||
+ missing)
|
||||
.max(1),
|
||||
}
|
||||
})
|
||||
@@ -239,8 +265,9 @@ pub fn fit_measure(
|
||||
anyhow::ensure!(
|
||||
width <= MAX_FEATURES,
|
||||
"fitting {target:?} would encode {width} features from {} records, which is too wide \
|
||||
to search on the UI thread. A category with a distinct value per record becomes one \
|
||||
column per record — mark such a category as a Label so it is not a feature",
|
||||
to search on the UI thread. A category becomes one column per distinct item, whether \
|
||||
its items read as numbers or not — mark such a category as a Label so it is not a \
|
||||
feature",
|
||||
table.rows.len(),
|
||||
);
|
||||
|
||||
@@ -264,13 +291,13 @@ pub fn fit_measure(
|
||||
cannot spell and a search cannot minimise. Correct or clear those cells first"
|
||||
);
|
||||
|
||||
// The exact width, now that it is known. The preflight above is advisory, and
|
||||
// this is the check that enforces the bound in both directions of a drift.
|
||||
// The exact width, now that it is known. It bounds the error MESSAGE, not the
|
||||
// cost: `preprocess` has already run — the preflight is the cost bound (§14).
|
||||
anyhow::ensure!(
|
||||
data.feature_names.len() <= MAX_FEATURES,
|
||||
"fitting {target:?} encoded {} features, over the {MAX_FEATURES} this host will \
|
||||
search. A category with many distinct values becomes one column per value — \
|
||||
mark such a category as a Label so it is not a feature",
|
||||
search. A category becomes one column per distinct item, whether its items read as \
|
||||
numbers or not — mark such a category as a Label so it is not a feature",
|
||||
data.feature_names.len(),
|
||||
);
|
||||
// Empty is a **panic**, not a poor fit: zero features makes `random_tree` draw
|
||||
@@ -714,6 +741,184 @@ mod tests {
|
||||
(model, view)
|
||||
}
|
||||
|
||||
/// Bug improvise-x1ns (2nd): `preprocess` infers Date BEFORE Categorical and
|
||||
/// `build_date` emits 5 or 7 parts. Evidence: `context/test-evidence/gp.md`.
|
||||
#[test]
|
||||
fn the_preflight_counts_the_parts_a_date_measure_expands_into() {
|
||||
for (label, stamp, parts) in [
|
||||
("a plain date", "2024-01-15", 5),
|
||||
("a date with a time", "2024-01-15T09:30:00", 7),
|
||||
] {
|
||||
let (model, view) = a_date_valued_measure(stamp);
|
||||
let table = feature_table(&model, &view, "Score").expect("the table builds");
|
||||
let category_columns = model.regular_category_names().len();
|
||||
|
||||
// The fixture must really be the shape the bug needs: ONE distinct
|
||||
// date, which the categorical arm would count as a single column.
|
||||
let when: Vec<&str> = table
|
||||
.rows
|
||||
.iter()
|
||||
.map(|r| r[category_columns].as_str())
|
||||
.collect();
|
||||
assert!(
|
||||
when.iter().all(|c| *c == stamp),
|
||||
"{label}: every When cell must be the same date: {when:?}"
|
||||
);
|
||||
assert!(
|
||||
gp_tree::data::parse_datetime(stamp).is_some_and(|d| d.has_time == (parts == 7)),
|
||||
"{label}: the fixture must really carry a time iff {parts} parts are expected"
|
||||
);
|
||||
|
||||
let width = super::encoded_width(&table, category_columns);
|
||||
let overrides =
|
||||
super::as_categories(&table.header, category_columns).expect("the header is clean");
|
||||
let data = preprocess(&table.header, &table.rows, &overrides);
|
||||
|
||||
assert_eq!(
|
||||
data.feature_names
|
||||
.iter()
|
||||
.filter(|n| n.starts_with("When."))
|
||||
.count(),
|
||||
parts,
|
||||
"{label}: the fixture must really expand into {parts} date parts — {:?}",
|
||||
data.feature_names
|
||||
);
|
||||
assert!(
|
||||
width >= data.feature_names.len(),
|
||||
"{label}: the preflight under-reported: it promised {width} features \
|
||||
and `preprocess` materialised {} — {:?}",
|
||||
data.feature_names.len(),
|
||||
data.feature_names,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// One regular category and a `When` MEASURE whose every cell is the same
|
||||
/// timestamp — the low-cardinality date column a distinct-count misses.
|
||||
fn a_date_valued_measure(stamp: &str) -> (Model, View) {
|
||||
let mut model = Model::new("sales");
|
||||
model.add_category(REGION).expect("under the limit");
|
||||
let measure = model.category_mut(MEASURE).expect("_Measure always exists");
|
||||
for item in ["When", "Score"] {
|
||||
measure.add_item(item);
|
||||
}
|
||||
for (i, region) in ["North", "South", "East"].iter().enumerate() {
|
||||
model
|
||||
.category_mut(REGION)
|
||||
.expect("just added")
|
||||
.add_item(*region);
|
||||
model.set_cell(
|
||||
CellKey::new(vec![
|
||||
(REGION.to_string(), (*region).to_string()),
|
||||
(MEASURE.to_string(), "When".to_string()),
|
||||
]),
|
||||
CellValue::Text(stamp.to_string()),
|
||||
);
|
||||
model.set_cell(
|
||||
CellKey::new(vec![
|
||||
(REGION.to_string(), (*region).to_string()),
|
||||
(MEASURE.to_string(), "Score".to_string()),
|
||||
]),
|
||||
CellValue::Number(i as f64),
|
||||
);
|
||||
}
|
||||
let mut view = View::new("Default");
|
||||
for cat in model.category_names() {
|
||||
view.on_category_added(cat);
|
||||
}
|
||||
view.set_axis(REGION, Axis::Row);
|
||||
view.set_axis(MEASURE, Axis::Page);
|
||||
(model, view)
|
||||
}
|
||||
|
||||
/// Bug improvise-x1ns: the preflight counted a forced categorical by its
|
||||
/// PRESENT values only. Evidence: `context/test-evidence/gp.md`.
|
||||
#[test]
|
||||
fn the_preflight_counts_the_missing_category_a_forced_column_materialises() {
|
||||
let (model, view) = a_numeric_category_with_a_missing_coordinate();
|
||||
let table = feature_table(&model, &view, "Score").expect("the table builds");
|
||||
let category_columns = model.regular_category_names().len();
|
||||
|
||||
// The fixture must really carry the shape the bug needs: a forced
|
||||
// category whose cells are numeric, one of them absent.
|
||||
let year: Vec<&str> = table.rows.iter().map(|r| r[0].as_str()).collect();
|
||||
assert_eq!(
|
||||
year.iter().filter(|c| gp_tree::data::is_missing(c)).count(),
|
||||
1,
|
||||
"exactly one record must lack the Year coordinate: {year:?}"
|
||||
);
|
||||
assert!(
|
||||
year.iter()
|
||||
.filter(|c| !gp_tree::data::is_missing(c))
|
||||
.all(|c| c.parse::<f64>().is_ok()),
|
||||
"the present Year cells must all be numeric: {year:?}"
|
||||
);
|
||||
|
||||
let width = super::encoded_width(&table, category_columns);
|
||||
let overrides =
|
||||
super::as_categories(&table.header, category_columns).expect("the header is clean");
|
||||
let data = preprocess(&table.header, &table.rows, &overrides);
|
||||
|
||||
// `build_categorical` maps the absent cell to the reserved `(missing)`
|
||||
// category, so it is one more one-hot column than the present values.
|
||||
assert!(
|
||||
data.feature_names
|
||||
.iter()
|
||||
.any(|n| n == &format!("Year={}", gp_tree::data::MISSING_CAT)),
|
||||
"the forced column must really emit a (missing) one-hot: {:?}",
|
||||
data.feature_names
|
||||
);
|
||||
assert!(
|
||||
width >= data.feature_names.len(),
|
||||
"the preflight under-reported: it promised {width} features and \
|
||||
`preprocess` materialised {} — {:?}",
|
||||
data.feature_names.len(),
|
||||
data.feature_names,
|
||||
);
|
||||
}
|
||||
|
||||
/// A numeric `Year` category with one record that has no `Year` coordinate, so
|
||||
/// forcing sends a numeric column down the categorical branch with a gap in it.
|
||||
fn a_numeric_category_with_a_missing_coordinate() -> (Model, View) {
|
||||
let mut model = Model::new("sales");
|
||||
model.add_category("Year").expect("under the limit");
|
||||
let measure = model.category_mut(MEASURE).expect("_Measure always exists");
|
||||
for item in ["Cost", "Score"] {
|
||||
measure.add_item(item);
|
||||
}
|
||||
for i in 0..4u32 {
|
||||
let year = format!("{}", 2020 + i);
|
||||
model
|
||||
.category_mut("Year")
|
||||
.expect("just added")
|
||||
.add_item(year.clone());
|
||||
for (m, v) in [("Cost", f64::from(i) * 10.0), ("Score", f64::from(i))] {
|
||||
model.set_cell(
|
||||
CellKey::new(vec![
|
||||
("Year".to_string(), year.clone()),
|
||||
(MEASURE.to_string(), m.to_string()),
|
||||
]),
|
||||
CellValue::Number(v),
|
||||
);
|
||||
}
|
||||
}
|
||||
// The record with no `Year` coordinate at all — an empty cell, which
|
||||
// `is_missing` reads as absent rather than as a value.
|
||||
for (m, v) in [("Cost", 99.0), ("Score", 9.0)] {
|
||||
model.set_cell(
|
||||
CellKey::new(vec![(MEASURE.to_string(), m.to_string())]),
|
||||
CellValue::Number(v),
|
||||
);
|
||||
}
|
||||
let mut view = View::new("Default");
|
||||
for cat in model.category_names() {
|
||||
view.on_category_added(cat);
|
||||
}
|
||||
view.set_axis("Year", Axis::Row);
|
||||
view.set_axis(MEASURE, Axis::Page);
|
||||
(model, view)
|
||||
}
|
||||
|
||||
/// The review finding on improvise-zv8t: forcing is by NAME and the preflight
|
||||
/// counts by INDEX. Evidence: `context/test-evidence/gp.md`.
|
||||
#[test]
|
||||
|
||||
@@ -812,6 +812,70 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// improvise-e8no declines ordinal encoding on semantics, NOT on
|
||||
/// spellability — a first draft said otherwise. `context/code/gp.md`.
|
||||
#[test]
|
||||
fn an_ordinal_split_on_a_category_is_spellable_as_an_indicator_sum() {
|
||||
// `is_expr` is a `primary`, so it is an `Expr` yielding 1/0, and
|
||||
// `cmp_op`'s operands are full `expr`s.
|
||||
for text in [
|
||||
"Score = (Year IS |2020|) + (Year IS |2021|)",
|
||||
"Score = IF(2020*(Year IS |2020|) + 2021*(Year IS |2021|) < 2020.5, 1, 2)",
|
||||
] {
|
||||
assert!(
|
||||
parse_formula(text, MEASURE).is_ok(),
|
||||
"the grammar the decline reasons about must really accept {text:?}"
|
||||
);
|
||||
}
|
||||
// And the reason a numeric item name must be pipe-quoted: a bare
|
||||
// number is not a `filter_value`.
|
||||
assert!(
|
||||
parse_formula("Score = (Year IS 2020)", MEASURE).is_err(),
|
||||
"a bare number as a filter_value must stay a parse error"
|
||||
);
|
||||
}
|
||||
|
||||
/// improvise-e8no's decline rests on this: the pre-zv8t encoding of a
|
||||
/// numeric category yields fits no document can store. Evidence: `gp.md`.
|
||||
#[test]
|
||||
fn a_split_on_a_numeric_category_read_as_a_number_cannot_be_stored() {
|
||||
use gp_tree::data::{Forced, Overrides, preprocess};
|
||||
|
||||
// The pre-improvise-zv8t reading: a category of numbers inferred as a
|
||||
// numeric column. Asked of `preprocess`, never a fabricated spelling.
|
||||
let header = vec!["Year".to_string(), TARGET.to_string()];
|
||||
let rows: Vec<Vec<String>> = [("2020", "1"), ("2021", "2"), ("2022", "3")]
|
||||
.iter()
|
||||
.map(|(y, t)| vec![(*y).to_string(), (*t).to_string()])
|
||||
.collect();
|
||||
let overrides = Overrides {
|
||||
force: [("Year".to_string(), Forced::Numeric)]
|
||||
.into_iter()
|
||||
.collect(),
|
||||
..Overrides::default()
|
||||
};
|
||||
let data = preprocess(&header, &rows, &overrides);
|
||||
let features = crate::Features::of(&data, &["Year".to_string()]);
|
||||
|
||||
assert_eq!(
|
||||
data.feature_names,
|
||||
vec!["Year".to_string()],
|
||||
"the fixture must really encode Year as ONE numeric column"
|
||||
);
|
||||
assert_eq!(
|
||||
features.0[0].spelling,
|
||||
crate::Spelling::Artefact(Artefact::NumericCategory),
|
||||
"a numeric column over a category column is unspellable"
|
||||
);
|
||||
|
||||
let tree = split(0, 2020.5, leaf(1.0), leaf(2.0));
|
||||
let err = fitted_formula(&tree, &features, TARGET, MEASURE, &document())
|
||||
.expect_err("a split on a numeric category is not storable")
|
||||
.to_string();
|
||||
assert!(err.contains("Year"), "{err}");
|
||||
assert!(err.contains("cannot spell"), "{err}");
|
||||
}
|
||||
|
||||
/// The property the *lying* test could not state: if `fitted_formula` says
|
||||
/// `Ok`, no cell of the target is a `ref:` error. See the evidence file.
|
||||
#[test]
|
||||
|
||||
@@ -139,11 +139,9 @@ mod tests {
|
||||
assert_eq!(r.len(), 2, "comma split a field: {r:?}");
|
||||
}
|
||||
assert_eq!(&records[1][0], "Smith, Jr.");
|
||||
assert!(
|
||||
records[1][1].starts_with("1,200"),
|
||||
"expected comma-grouped value, got: {:?}",
|
||||
&records[1][1]
|
||||
);
|
||||
// Exact, not `starts_with`: the fixture asks for `.2f`, and a prefix
|
||||
// check cannot tell two decimals from none (improvise-711n).
|
||||
assert_eq!(&records[1][1], "1,200.00");
|
||||
}
|
||||
|
||||
/// Bug improvise-iot6: `render_csv` built its layout with `GridLayout::new`,
|
||||
|
||||
@@ -13,6 +13,13 @@ serde = { version = "1", features = ["derive"] }
|
||||
# (`AxisFrame::header_widths`). A terminal column is a display cell, not a
|
||||
# `char`, and the TUI has always measured it this way.
|
||||
unicode-width = "^0.2"
|
||||
# Only for `variants`, which is off unless a test build asks for it.
|
||||
serde_json = { version = "1", optional = true }
|
||||
|
||||
[features]
|
||||
# `variants` reaches OUT of this crate: improvise-command bounds its own
|
||||
# ModelCommand enumeration with it. Off in every shipped build (improvise-26dm).
|
||||
test-support = ["dep:serde_json"]
|
||||
|
||||
[dev-dependencies]
|
||||
serde_json = "1"
|
||||
|
||||
@@ -13,8 +13,8 @@ pub mod keymap;
|
||||
mod reduce;
|
||||
pub mod render;
|
||||
pub mod resolve;
|
||||
#[cfg(test)]
|
||||
pub(crate) mod variants;
|
||||
#[cfg(any(test, feature = "test-support"))]
|
||||
pub mod variants;
|
||||
pub mod viewmodel;
|
||||
|
||||
pub use basis::{Basis, BasisCache, Rejection};
|
||||
|
||||
Reference in New Issue
Block a user