Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deviser tests documentation #41

Merged
merged 64 commits into from
Jan 21, 2021
Merged
Changes from 1 commit
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
d069815
A little simplification.
Dec 15, 2020
6ea87b2
Documenting functions in run_matlab_tests.py.
Dec 15, 2020
31103f5
Tidyin gup a bit and adding function docstring.
Dec 15, 2020
2985780
A bit more tidying.
Dec 15, 2020
7129b69
Replace global_variables with gv.
Dec 15, 2020
9d328a5
Started documenting test_functions.py.
Dec 15, 2020
08b3d3b
Added more docstrings to test_functions.py.
Dec 15, 2020
a6818ca
A little bit of tidying up.
Dec 15, 2020
2c37c22
Took out some common code.
Dec 16, 2020
4fd9bc7
Using os.path.normpath() means a file path should work on both Window…
Dec 16, 2020
70abf42
Now we can run more tests.
Dec 16, 2020
92e97b4
More meaningful function name. Some tidying.
Dec 17, 2020
f128fa1
I've added docstrings to the functions I understand.
Dec 17, 2020
84a3ac8
Final comment for now.
Dec 17, 2020
74f1260
Using os.path.normpath() means this should work on Windows and *nix.
Dec 17, 2020
ec85fe2
Extract common code into setup().
Dec 17, 2020
cdd8a3f
Now we have a common compare function.
Dec 17, 2020
32a8e3c
More generic functions to replace specifics.
Dec 17, 2020
01fc802
Finished documenting run_text_tests.py.
Dec 17, 2020
1071fac
Adding os.path.normpath() lets us run the tsts on Windows and *nix.
Dec 17, 2020
bc23d8f
Extracted common set-up code into its own function.
Dec 17, 2020
b15acc2
Removed wrapper function around test_functions.compare_files().
Dec 17, 2020
bd23dd7
Added some docstrings.
Dec 17, 2020
f2cae72
Added what docstrings I can.
Dec 17, 2020
503e097
Added more text and TODO statements.
Jan 5, 2021
c546a49
Used os.path.normpath() for portability.
Jan 6, 2021
d17e055
Finished most of the basic documentation for run_cmake_tests.py.
Jan 6, 2021
a3c5485
Fixed some of the flake8 complaints.
Jan 6, 2021
abe41fb
Added more os.path.normpath() statements.
Jan 6, 2021
da46fb9
Added more os.path.normpath() statements to run_examples_tests.py, bu…
Jan 6, 2021
31ca25c
Added some os.path.normpath() statements to run_matlab_tests.py
Jan 6, 2021
5731835
Added os.path.normpath() statements to run_cpp_tests.py
Jan 6, 2021
57f005c
Moving always-run tests to beginning of test set.
Jan 6, 2021
3a81c58
Abstracted some common code into generate_generic_header().
Jan 6, 2021
a39cf9f
Extracted common code into common_setup().
Jan 6, 2021
5e215b9
More function documenting in run_cpp_tests.py
Jan 6, 2021
d2461db
Moving common code into new version of compare_files().
Jan 6, 2021
36fc367
Added more documentation to the comparison functions.
Jan 6, 2021
edbd39d
Added docs to the test runner functions.
Jan 6, 2021
56315bf
Ran all cpp test files through dos2unix so python 2 tests will succeed.
Jan 6, 2021
e45e104
Added explanatory note about dos2unix etc.
Jan 6, 2021
61b8ad5
Fixed some typos and flake8 complaints.
Jan 7, 2021
32e6dbb
Added os.path.normpath() statements to make the code portable.
Jan 7, 2021
1efd1ca
Moved a lot of duplicate code into new function common_set_up().
Jan 7, 2021
82abddd
Moved some common code into new function go_into_new_directory().
Jan 7, 2021
9844227
Took out some more common code.
Jan 7, 2021
4ee0b16
Took out some more common code into new comparison functions.
Jan 7, 2021
6ec4c25
import global_variables as gv
Jan 7, 2021
8603651
Fixed a bug. Noted some failed and skipped tests.
Jan 7, 2021
2e36afc
Stopped most tests skipping.
Jan 7, 2021
3270d82
Now we are no longer skipping tests.
Jan 7, 2021
bbe51fb
Now we are missing only two files.
Jan 7, 2021
03d9979
Now no files are missing.
Jan 11, 2021
2a8e90d
Started documenting run_other_library_tests.py
Jan 11, 2021
d59fd67
Removed unused function parameter.
Jan 11, 2021
9035473
Added more docstrings, and also queries.
Jan 11, 2021
15d895a
I think this is as much as I can do with this file.
Jan 12, 2021
e119943
added some explanations
skeating Jan 15, 2021
80f3d34
added explanation
skeating Jan 15, 2021
9ee5153
oops forgot to change param in code
skeating Jan 15, 2021
3c4dc4e
note that files are not yet part of testing
skeating Jan 15, 2021
40b3149
Adding Sarah's PR comments etc.
Jan 18, 2021
4c48705
Merge branch 'develop' into matt-tests
Jan 18, 2021
6b82a4f
Hopefully fixing the last of Sarah's required changes for PR 41
Jan 19, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Hopefully fixing the last of Sarah's required changes for PR 41
  • Loading branch information
Matthew Gillman committed Jan 19, 2021
commit 6b82a4f71192bb1220ed539aac3668eb394db5ce
2 changes: 1 addition & 1 deletion generator/code_files/ValidationFiles.py
Original file line number Diff line number Diff line change
@@ -47,7 +47,7 @@


class ValidationFiles():
"""Class for all validation-related files TODO Is it? """
"""Class for all validation-related files"""

def __init__(self, lib_object, verbose=False):
# members from object
80 changes: 35 additions & 45 deletions generator/tests/test_binding_code/run_bindings_tests.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,28 @@
#!/usr/bin/env python
"""
Generating the binding files creates a number of files for each language,
as well as swig and local specific files.

Every language will generate a file to downcast extensions, namespaces and
specific classes.

e.g. generating csharp bindings for package 'spatial' will result in files:

bindings
csharp
local-downcast-extension-spatial.i
local-downcast-namespaces-spatial.i
local-packages-spatial.i
swig
spatial-packages.h
spatial-packages.i

Certain languages will also require .cpp files.

Testing these involves creating each of these specific files for a
language/package combination and having distinct functions to create and
compare the files.
"""

import os
import sys
@@ -10,12 +34,14 @@

import test_functions


##############################################################################
# Set up variables
fails = []
not_tested = []



##############################################################################
# Specific generation functions

@@ -55,7 +81,7 @@ def generate_bindings_downcast_ext(filename, binding):

def generate_bindings_downcast_ns(filename, binding):
"""
Generate a downcast namespace. TODO such as?
Generate a downcast namespace.
"""
all_files = set_up(filename, binding)
all_files.write_downcast_namespace()
@@ -64,7 +90,7 @@ def generate_bindings_downcast_ns(filename, binding):

def generate_bindings_downcast_pkgs(filename, binding, local):
"""
Generate a downcast package. TODO such as?
Generate a downcast package.

:param filename:
:param binding:
@@ -82,7 +108,7 @@ def generate_bindings_downcast_pkgs(filename, binding, local):

def generate_bindings_downcast_plugins(filename, binding):
"""
Generate a downcast plugin. TODO such as?
Generate a downcast plugin.
"""
all_files = set_up(filename, binding)
all_files.write_downcast_plugins()
@@ -106,16 +132,6 @@ def compare_code(name, binding, ext):
not_tested)


def compare_ext_headers(class_name):
"""
TODO This function doesn't appear to be used anywhere.
"""
correct_file = os.path.normpath('./test-extension/{0}.h'.format(class_name))
temp_file = os.path.normpath('./temp/{0}.h'.format(class_name))
return test_functions.compare_files(correct_file, temp_file, fails,
not_tested)


#############################################################################
# Specific test functions

@@ -138,35 +154,9 @@ def run_ns_test(name, binding, ext):
return fail


# TODO I'm not sure what these different test functions are testing, exactly.
"""
Generating the binding files creates a number of files for each language,
as well as swig and local specific files.

Every language will generate a file to downcast extensions, namespaces and
specific classes.

e.g. generating csharp bindings for package 'spatial' will result in files:

bindings
csharp
local-downcast-extension-spatial.i
local-downcast-namespaces-spatial.i
local-packages-spatial.i
swig
spatial-packages.h
spatial-packages.i

Certain languages will also require .cpp files.

Testing these involves creating each of these specific files for a
language/package combination and having distinct functions to create and
compare the files.
"""

def run_test(name, binding, ext):
"""
Run a test. TODO Sarah please expand...
Run a test.

:param name: package name
:param binding: the required binding, e.g. 'csharp', 'java', etc.
@@ -185,7 +175,7 @@ def run_test(name, binding, ext):

def run_pkgs_test(name, binding, ext):
"""
Run a pkgs test. TODO Sarah please expand...
Run a pkgs test.

:param name: package name
:param binding: the required binding, e.g. 'csharp', 'java', etc.
@@ -207,12 +197,12 @@ def run_pkgs_test(name, binding, ext):

def run_local_test(name, binding, ext, local):
"""
Run a local test. TODO Sarah please expand...
Run a local test.

:param name: package name
:param binding: the required binding, e.g. 'csharp', 'java', etc.
:param ext: the file extension
:param local: TODO Sarah please expand...
:param local:
:return: 0 on success or missing file, 1 on failure.
"""
filename = test_functions.set_up_test(name, 'local', binding)
@@ -228,7 +218,7 @@ def run_local_test(name, binding, ext, local):

def run_plugin_test(name, binding, ext):
"""
Run a plugin test. TODO Sarah please expand...
Run a plugin test.

:param name: package name
:param binding: the required binding, e.g. 'csharp', 'java', etc.
@@ -245,7 +235,7 @@ def run_plugin_test(name, binding, ext):

def run_swig_test(name, binding, ext):
"""
Run a SWIG test. TODO Sarah please expand...
Run a SWIG test.

:param name: package name
:param binding: the required binding, e.g. 'csharp', 'java', etc.
19 changes: 5 additions & 14 deletions generator/tests/test_cmake_code/run_cmake_tests.py
Original file line number Diff line number Diff line change
@@ -96,25 +96,16 @@ def compare_files(correct_file, temp_file):
not_tested)


def compare_cmake(name, is_src=False):
def compare_cmake(name): # , is_src=False):
"""
Do a string comparison of the contents of two files.

TODO: the two cases checked by this function are identical!
Presumably a mistake?

:param name: TODO
:param is_src: TODO
:return: 0 on success, or file not present; 1 on failure.
"""
if is_src:
correct_file = os.path.normpath('./test-cmake/{0}-package.cmake'.
format(name))
temp_file = os.path.normpath('./temp/{0}-package.cmake'.format(name))
else:
correct_file = os.path.normpath('./test-cmake/{0}-package.cmake'.
format(name))
temp_file = os.path.normpath('./temp/{0}-package.cmake'.format(name))
correct_file = os.path.normpath('./test-cmake/{0}-package.cmake'.
format(name))
temp_file = os.path.normpath('./temp/{0}-package.cmake'.format(name))
return compare_files(correct_file, temp_file)


@@ -180,7 +171,7 @@ def run_cmake_test(name):
filename = test_functions.set_up_test(name, 'CMake')
generate_cmake_files(filename)
fail = compare_cmake(name)
fail += compare_cmake(name, True)
# fail += compare_cmake(name, True)
print('')
return fail

15 changes: 10 additions & 5 deletions generator/tests/test_cpp_code/run_cpp_tests.py
Original file line number Diff line number Diff line change
@@ -38,9 +38,10 @@ def generate_new_cpp_header(filename, num):
Generate cpp files.

:param filename: name of XML file to parse.
:param num: TODO seems to be index of item in list which is the value
:param num: index of item in list which is the value
(in the big dictionary structure) for the key 'baseElements'.
But I don't fully understand this.
It is the index so that we can do code for just one class and
not whole of the object returned from ParseXML
:return: nothing
"""
ob = common_setup(filename)
@@ -238,7 +239,8 @@ def run_plug_test(name, class_name, test_case, num):
:param name: which set of tests this relates to, e.g. 'qual'
:param class_name: C++ class name/start of .h and .cpp file names, e.g. 'QualModelPlugin'
:param test_case: brief description, e.g. 'basic plugin'
:param num: plugin #. TODO I'm not too sure about this. Please can we have an example?
:param num: plugin #. It's just the index in the list of plugins that are
part of large dict object form ParseXML
:return: number of failed tests.
"""
filename = test_functions.set_up_test(name, class_name, test_case)
@@ -731,8 +733,11 @@ def main():
test_case = 'sidref with multiple targets'
fail += run_valid_test(name, class_name, test_case)

# TODO Sarah, I'm guessing maybe the next few tests are iterating over something?
# Because it's not clear; all these "diff spacing" tests look identical.
# These next tests are testing the spacing of the element content
# eg.
# element="ThingA, ThingB"
# element=" ThingA, ThingB"
# element="ThingA,ThingB"

name = 'test_sidrefs_1'
class_name = 'RefsSBMLError'
5 changes: 3 additions & 2 deletions generator/tests/test_examples/run_examples_tests.py
Original file line number Diff line number Diff line change
@@ -72,7 +72,8 @@ def generate_xml(filename):

def generate_xml_fails(filename):
"""
Generate a failure example?????? TODO unsure
Generate a failure example (creates an xml
file that would fail validation)

:param filename: XML file to parse
:return: package name, e.g. 'dyn'
@@ -94,7 +95,7 @@ def generate_some_xml_fails(filename, start, stop, number=-1):
:param number: number of files to write? TODO not sure.
:return: package name, e.g. 'dyn'

TODO Sorry Sarah I'm a bit confused by this one!
Sarah: "This was a work in progress and is not included in overall test run"
"""
ob = set_up(filename)
all_files = ValidationXMLFiles.ValidationXMLFiles(ob)
16 changes: 12 additions & 4 deletions generator/tests/test_other_library/run_other_library_tests.py
Original file line number Diff line number Diff line change
@@ -59,7 +59,8 @@ def generate_new_cpp_header(filename, num):
Generate new C++ header and code files.

:param filename: the XML file to parse.
:param num: TODO not too sure about this - it's a list index.
:param num: index allowing code to generate for just one class rather
than all - it's a list index.
:return: nothing
"""
ob = common_set_up(filename)
@@ -74,11 +75,17 @@ def generate_new_cpp_header(filename, num):

def generate_templates(filename):
"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several of the files for other libraries are generated using templates with appropriate names etc added.
See generator/code_files/templates
This function tests these

TODO not sure what this does. Not sure what ValidationFiles class is for.
And what do we mean by 'templates' here?
Several of the files for other libraries are generated using
templates with appropriate names etc added. See
generator/code_files/templates.
This function tests these

:param filename: the XML file to parse.
:return: nothing

Validation class causes the errors for things to be enumerated and strings
added. These will be needed by templates so with testing we have to
create them first
"""
ob = common_set_up(filename)
gv.populate_error_list(ob)
@@ -93,7 +100,8 @@ def generate_templates(filename):

def generate_common_templates(filename):
"""
Write the 'common' files. TODO need more explanation.
Write the 'common' files. These are the files that will be part of
the common directory as they are used by all files in library.

:param filename: the XML file to parse.
:return: nothing
2 changes: 1 addition & 1 deletion sphinx-docs/conf.py
Original file line number Diff line number Diff line change
@@ -23,7 +23,7 @@
project = 'generator'
copyright = '2020, Sarah Keating et al'
author = 'Sarah M. Keating, Frank T. Bergmann, Michael Hucka,' \
'Brett G. Olivier, Lucian P. Smith'
'Brett G. Olivier, Lucian P. Smith, Matthew S. Gillman'


# -- General configuration ---------------------------------------------------