v24.7.1

Features

• Automatically save the JSON Validation schema by @mialy-defelice in #1415
• feat: surfaced schematic exceptions in flask APIs by @linglp in #1438
• feat: added tracing to schematic code base for manifest generate and manifest submission by @linglp in #1430

Bug Fixes

• Bugfix: FDS-1686 test manifest being changed by @andrewelamb in #1412
• BugFix: Manifest version increment by @GiaJordan in #1435
• BugFix: Files no longer added to records based manifests by @GiaJordan in #1447

Technical Debt

• Develop FDS 243 mypy schema module by @andrewelamb in #1416
• added pydantic to toml file by @andrewelamb in #1419
• update pandas and numpy by @andrewelamb in #1421
• FDS-511 removed typing extensions lock by @andrewelamb in #1432
• technical debt: Remove service_acct_creds_synapse_id from schematic config by @linglp in #1436
• Add maintainers and version info to AWS Docker image by @GiaJordan in #1437
• technical debt: run black for schematic api and test and modify pre-commit by @linglp in #1442
• technical debt: added unit test for changeFileMetaData function by @linglp in #1444
• fix: added black compatibility to isort by @linglp in #1453
• technical debt: added isort to pre-commit hook in schematic repo by @linglp in #1441

Full Changelog: v24.5.1...v24.7.1

v24.5.1

Bug Fix

• BugFix: JSON Schema Validation Issues by @mialy-defelice in #1420
• technical debt: remove release.md from repo by @linglp in #1426
• Bug Fix: Fix failing test to conform to data model updates by @mialy-defelice in #1428

Full Changelog: v24.4.1...v24.5.1

v24.4.1

Features

• Allow non-entry if Required=False, while still enforcing validation rules if value is entered by @mialy-defelice in #1390
• Add Manifest download to the CLI by @mialy-defelice in #736
• Cross Manifest Validation: Allow first manifest to submit, if no target manifest to validate against. by @mialy-defelice in #1398
• Reorganize validation message level determination as a precursor to single rule disjunction by @GiaJordan in #1366
• Match None Manifest Validation Rule FDS-1766 by @mialy-defelice in #1387
• Add functionality to set requirements based per component by @mialy-defelice in #1401
• Updated python client to ^4.1.0 by @andrewelamb in #1403

Bug fixes

• Updated workflows to make sure uwsgi gets installed when publishing an image by @linglp in #1377
• Updated workflow to install dependencies and root project together by @linglp in #1380
• Schematic v24.2.1 AWS deployment fix by @linglp in #1378
• Schematic v24.2.1 AWS deployment fix - 2 by @linglp in #1382
• Fix wrong function name reference in function get_viz_component_attributes_explorer by @linglp in #1384
• Added installation of uWSGI in docker file by @linglp in #1381
• Avoid repeatedly calling generate_data_model_graph by @linglp in #1374
• Fix check_synapse_cache_size function; allow file size to be float by @linglp in #1389
• Added help text for file-annotations-upload parameter by @linglp in #1406
• Remove schematic init from CLI by @linglp in #1400
• Added project scope parameter in test_api.py by @linglp in #1409
• JSONLD dictionary parsing error by @mialy-defelice in #1407
• Convert nans in manifest to empty strings just prior to JSON Schema Validation by @mialy-defelice in #1402

Technical debt

• Mypy visualization module by @andrewelamb in #1373

• Mypy utils files by @andrewelamb in #1375

• Pylint schema utils by @andrewelamb in #1371

• Add pre commit hook for black by @GiaJordan in #1379

• Remove makrupsafe from toml file by @andrewelamb in #1383

• Remove temp data model references from test_api by @mialy-defelice in #1386

• Develop mypy schema utils by @andrewelamb in #1392

• Add synapse cache as a parameter to SynapseStorage by @andrewelamb in #1394

• Added file-annotations-upload parameter when submitting a manifest by @linglp in #1395

• Develop pylint schema module by @andrewelamb in #1393

• Turn pyopenssl package optional by @linglp in #1413

• Remove uneeded parameter from submit_metadata_manifest by @andrewelamb in #1410

• Schematic v24.4.1 by @lakikowolfe in #1417

New Contributors

• @lakikowolfe made their first contribution in #1417

Full Changelog: v24.2.1...v24.4.1

v24.2.1

What's Changed

Features:

• Retain formatting of Display name for annotations and table column names by @mialy-defelice in #1330
• Data Model Labels, display_label or class_label options by @mialy-defelice in #1356
• Component Based Rule Settings by @mialy-defelice in #1368

Bug Fixes

• feature-configuration-fixes by @andrewelamb in #1357
• Fixed an issue where empty rows would not be removed from manifests, and add test for load_df util function by @GiaJordan in #1346
• Increase schematic API time out by @linglp in #1358
• Deleted old version of readthedocs yaml file by @linglp in #1365

Technical Debt

• Updated poetry version in readthedocs.yml by @linglp in #1364
• Update pandas and schematic_db dependencies by @GiaJordan in #1347
• Run rule combinations tests at every push instead of only selectively by @GiaJordan in #1363
• Linted visualization module by @andrewelamb in #1359
• Move remained logic in routes.py related to manifest generator to library by @linglp in #1351
• Update parallel_map to parallel_applymap by @mialy-defelice in #1372
• Schematic v24.2.1 by @GiaJordan in #1376
• Make api packages optional by @andrewelamb in #1362
• Linted utils module by @andrewelamb in #1361
• Ran black on all modules by @andrewelamb in #1355
• Update synapse client minimum version to 3.2.0 by @GiaJordan in #1345

Full Changelog: v24.1.1...v24.2.1

Schematic Release (v24.1.1)

What's Changed

We have added a Data Model Validator that is currently run during the convert step to check that some key assumptions about the data model are fulfilled, and that is converted properly to the graph network. We have plans to expand the validator in the future and allow it to be run directly.
Users can now supply the CSV data model itself as the data model, instead of the JSONLD.
Breaking Changes: Users will be expected to regenerate their JSON-LD using the new refactor release. Previously generated JSON-LDs will not currently work with the refactored schematic.
Other Changes:

• Schema Refactor: Unit Tests for TestDataModelGraph by @mialy-defelice in #1309
• Tests for DataModelRelationships class by @andrewelamb in #1293
• Schema Refactor: Develop unit tests DataModelJsonSchema by @mialy-defelice in #1307
• Schema Refactor: Unit Tests for DataModelJSONLDParser by @mialy-defelice in #1308
• Refactor Schemas: Unit tests for DataModelJSONLD FDS-1064 by @mialy-defelice in #1313
• Refactor Schemas: Response to review comments to original PR by @mialy-defelice in #1315
• Schema Refactor: Unit Tests for DataModelParser by @mialy-defelice in #1304
• Refactor Schemas: unit tests FDS-1061 TestDataModelNodes by @mialy-defelice in #1310
• Schema Refactor: Tests for Data Model CSV Parser by @mialy-defelice in #1305
• Schema Refactor: Create tests for DataModelEdges by @GiaJordan in #1306
• Changes to address JSONLD processing by @mialy-defelice in #1321
• Revert changes related to workflow by @linglp in #1328
• Refactor Schemas: Initial Release BugFix, incorrect name causing issues with Manifest generation -- FDS-1442 by @mialy-defelice in #1338
• Revert "Refactor Schemas: Initial Release BugFix, incorrect name causing issues with Manifest generation -- FDS-1442" by @mialy-defelice in #1343
• Re-add change to change parentOf to subClassOf by @mialy-defelice in #1344
• Refactor Schemas: JSONLD Data Model Parsing, change from using labels to display names, merge develop by @mialy-defelice in #1348
• Refactor Schemas - Latest with Fixes. by @mialy-defelice in #1350
• Schematic Schemas Refactor: DM Parser by @mialy-defelice in #1126
• Schematic release v24.1.1 by @linglp in #1354
  Bug Fixes:
• Bug fix: Added missing import statements by @linglp in #1327
• Bug Fix: Fix issue with trailing comma creating an empty node by @mialy-defelice in #1329
• Bug fix: Fixed tests in test_api.py by @linglp in #1352

Full Changelog: v23.12.1...v24.1.1

Schematic release (v23.12.1)

What's Changed

Bug fixes

• Update dockerfile poetry versions FDS-1325 by @GiaJordan in #1325
• Updated poetry version when building schematic docker image for AWS deployment by @linglp in #1326
• Fixed typing that API uses by @andrewelamb in #1332
• Added fit=true parameter when setting a dataframe by @linglp in #1318
• Resolved issues with cross-manifest validation that occured when a manifest was only a single row by @GiaJordan in #1337
• Resolved an issue where a visualization endpoint intermittently returned the wrong information by @GiaJordan in #1336
• Make access token optional to fix CLI manifest submission by @mialy-defelice in #1340
• Remove great_expectations/expectations/Manifest_test_suite.json before running validation tests by @linglp in #1342

Features

• Reduce execution time for Test Schematic workflow by @GiaJordan in #1320

Others

• Move methods of creating multiple manifests from API to manifest generator by @linglp in #1333
• Update deprecated GX v0.15.x functions by @GiaJordan in #1335
• Update manifest unit test for new manifest behavior by @GiaJordan in #1334
• Update pyproject.toml notation by @GiaJordan in #1331

Full Changelog: v23.11.1...v23.12.1

v23.11.1

What's Changed

Bug Fixes:

• [BugFix]: pypi publish workflow (v23.9.3) by @andrewelamb in #1301
• [BugFix]: Addresses HTAN-250 & HTAN-258 issues: Add functions to return a display name or requirement if not available. by @mialy-defelice in #1303
• [BugFix]: Address an issue where the asset view was not being set correctly for operations involving manifest validation by @GiaJordan in #1312

New Features:

• Raise a helpful error when a datatype cannot be found in a schema by @GiaJordan in #1302
• Move API access_token parameter to request header by @GiaJordan in #1288
• Add access_token to /model/validate header by @GiaJordan in #1311
• Update store/synapse.updateDB to take sg and se by @mialy-defelice in #1291
• Add default value for access_token to another validation method FDS-1248 by @GiaJordan in #1319

Technical Debt:

• Redid poetry install in publish workflow by @andrewelamb in #1299
• Update minimum Poetry version required to 1.3.0 FDS-1218 by @GiaJordan in #1316
• Add --durations flag to pytest by @GiaJordan in #1223
• Add Component as a required attribute to the example data model by @mialy-defelice in #1259
• Project scope manifest validation tests by @GiaJordan in #1314
• Update schematic dependencies FDS-1312 by @GiaJordan in #1322
• Schematic v23.11.1 by @GiaJordan in #1317

Full Changelog: v23.9.3...v23.11.1

v23.9.3

What's Changed

• BugFix: is_class_in_schema no longer errors out when a class is not in schema by @GiaJordan in #1287
• BugFix: Fix error when use_annotations=True for record based metadata where there are no existing annotations by @GiaJordan in #1285
• Bug Fix: Adding annotations to files with -mrt file_only parameter by @mialy-defelice in #1290
• Develop markers for api tests according to credentials required FDS-1026 by @GiaJordan in #1289

Full Changelog: v23.9.2...v23.9.3

v23.9.1

What's Changed

• [bug fix] Renamed function related to setting background color of required columns by @linglp in #1276
• [bug fix] Added includesType parameter when using walk function by @linglp in #1281
• [bug fix] Do not pull annotations when use_annotations set to False and there is no existing manifests by @linglp in #1278
• Add unit test for add_root_to_component function by @linglp in #1282
• [bug fix] Skipped submission when running API tests remotely by @linglp in #1280
• Refactored logic when pulling annotations by @linglp in #1279
• [bug fix] Fix issues when generating an existing manifest as an excel sheet by @linglp in #1284
• Schematic release 23.9.1 by @andrewelamb in #1283

Full Changelog: v23.8.1...v23.9.1

v23.8.1

What's Changed

• [bug fix] Updated docker file when building docker images for schematic AWS deployment by @linglp in #1269
• update channel id to fair-data-team channel by @GiaJordan in #1272
• [bug fix] Updated manifest download endpoint to avoid querying file view by @linglp in #1270
• [bug fix] Fixed typo in manifest generator by @linglp in #1275
• Add API endpoint that returns the current version of schematic by @GiaJordan in #1277
• [bug fix] Updated .synapseCache, functions to calculate cache, and cleared manifests before each download by @linglp in #1268
• replace issue templates by @allaway in #1274
• Schematic v23.8.1 FDS-807 by @GiaJordan in #1273

Full Changelog: v23.7.1...v23.8.1