Tutorial: using NeurEco Python API for a Tabular Regression problem#

The following section uses the test case Metamaterial Antennas. This test case is included in the NeurEco installation package.

Build a model#

  • Create an empty directory (MetamaterialAntenna Example), extract the Metamaterial Antennas test case data there. The created directory contains the following files:

    • x_test.csv

    • y_test.csv

    • x_train.csv

    • y_train.csv

  • Import the required libraries (NeurEco and NumPy):

from NeurEco import NeurEcoTabular as Tabular
import numpy as np

  • Load the training data:

x_train = np.genfromtxt("x_train.csv", delimiter=";", skip_header=True)
y_train = np.genfromtxt("y_train.csv", delimiter=";", skip_header=True)

  • Initialize a NeurEco object to handle the Regression problem:

builder = Tabular.Regressor()

All the methods provided by the Regressor class can be viewed by calling the __methods__ attributes:

print(builder.__methods__)

*** NeurEco Tabular Regressor methods: ***
- load
- save
- delete
- evaluate
- build
- get_input_count
- get_output_count
- load_model_from_checkpoint
- get_number_of_networks_from_checkpoint
- get_weights
- export_fmu
- export_c
- export_onnx
- export_vba
- compute_error
- plot_network
- forward_derivative
- gradient
- set_weights
- perform_input_sweep

To understand what each parameter of any method does and how to use it, print the doc of the method:

print(builder.export_c.__doc__)

exports a NeurEco tabular model to a header file
:param h_file_path: path where the .h file will be saved
:param precision: string: optional: "float" or "double": precision of the weights in the h file
:return: export_status: int: 0 if export is ok, other if otherwise.

builder.build(input_data=x_train, output_data=y_train,
          # the rest of these parameters are optional
          write_model_to="./MetamaterialAntennas/MetamaterialAntennas",
          checkpoint_address="./MetamaterialAntennas/MetamaterialAntennas.checkpoint",
          outputs_normalize_per_feature=True)

  • When build is called, NeurEco starts the building process:

Validation Percentage will be used to get the validation data. This is due to:
- one or all the validation data is set to None
- validation indices is set to None

 info >
 info >      _   __                ______
 info >     / | / /__  __  _______/ ____/________
 info >    /  |/ / _ \/ / / / ___/ __/ / ___/ __ \
 info >   / /|  /  __/ /_/ / /  / /___/ /__/ /_/ /
 info >  /_/ |_/\___/\__,_/_/  /_____/\___/\____/
 info >                  === A D A G O S ===
 info >
 info > Version: 4.01.2474.0 Compiled with MSVC v1928  Oct 12 2022 Matlab runtime:no
 info > OpenMP: yes
 info > MKL: yes
 info > Reading data files...
 info > Reading Data from C:/Users/Sadok/AppData/Local/Temp/tmp3wmwwjv4/inputs_tab_train.npy
 info > Reading Data from C:/Users/Sadok/AppData/Local/Temp/tmp3wmwwjv4/outputs_tab_train.npy
 info > build for: 6 outputs and 7 inputs and 457 samples.
 info > Building Model

During the build NeurEco saves the intermediate modes to the checkpoint file (defined by the parameter checkpoint_address). To load and use the intermediate models from this checkpoint:

  • Create a new NeurEco object in which to load the model:

model = Tabular.Regressor()

  • Determine how many intermediate models the checkpoint contains:

n = model.get_number_of_networks_from_checkpoint("./MetamaterialAntennas/MetamaterialAntennas.checkpoint")

  • Load any intermediate model from the checkpoint using its id (count starts with zero). For this example, at the moment of running the command \(n=6\), and the following command loads the intermediate model \(n°3 \ (id=2)\):

model.load_model_from_checkpoint("./MetamaterialAntennas/MetamaterialAntennas.checkpoint", 2)

Now model is a valid Regression model, and can be used as usual.

  • Check the number of trainable parameters each of the intermediate models has:

for i in range(n):
    print("Loading model", i, " from checkpoint file:")
    model.load_model_from_checkpoint("./MetamaterialAntennas/MetamaterialAntennas.checkpoint", i)
    print("number of trainable parameters in intermediate model --", i, " is:", model.get_weights().size)

Loading model 0  from checkpoint file:
number of trainable parameters in intermediate model -- 0  is: 34
Loading model 1  from checkpoint file:
number of trainable parameters in intermediate model -- 1  is: 62
Loading model 2  from checkpoint file:
number of trainable parameters in intermediate model -- 2  is: 90
Loading model 3  from checkpoint file:
number of trainable parameters in intermediate model -- 3  is: 132

Evaluate a model#

  • Load the testing data from the CSV files:

x_test = np.genfromtxt("x_test.csv", delimiter=";", skip_header=True)
y_test = np.genfromtxt("y_test.csv", delimiter=";", skip_header=True)

  • Create a Regressor object to use for the evaluation:

evaluator = Tabular.Regressor()

Note

It is possible to use the already existing Regressor object builder when the evaluation is done just after the build, and builder is still available.

  • Load the built model:

load_state = evaluator.load("./MetamaterialAntennas/MetamaterialAntennas")

Note

When building or evaluating a NeurEco model, all the used paths do not necessarily need to have an extension when they are passed as parameters to a NeurEco method.

  • To extract information from the loaded model, such as the number of inputs, the number of outputs and the weights array, run:

n_inputs = evaluator.get_input_count()
n_outputs = evaluator.get_output_count()
weights = evaluator.get_weights()
print("Number of Inputs:", n_inputs)
print("Number of Outputs:", n_outputs)
print("Number of trainable parameters:", weights.size)

Number of Inputs: 7
Number of Outputs: 6
Number of trainable parameters: 458

  • To plot the network graph (this operation requires matplotlib library installed, see Plot a NeurEco network):

evaluator.plot_network()
MetamaterialAntennaPythonNetworkPlot

Python API operations: plotting a network: test case - MetamaterialAntennas#

  • To evaluate the model on the test data:

neureco_outputs = evaluator.evaluate(x_test)
l2_error = evaluator.compute_error(neureco_outputs, y_test)
print("L2 relative error (%):", 100 * l2_error)

L2 relative error (%): 1.4545507588248332

Note

During evaluation, the normalization is carried out by the model and its parameters are not relative to the data set being evaluated, but are the global parameters computed during the build of the model.

  • To perform an input sweep (see Input sweep, this operation requires matplotlib library installed), run, for example:

evaluator.perform_input_sweep(x=x_test[49, :], input_id=0, input_interval=[2143, 6956.], output_id=0)
MetamaterialAntennaPythonInputSweep

Python API operations: Performing an input sweep: test case - MetamaterialAntennas#

  • To save the model in the native NeurEco binary format:

save_state = evaluator.save("MetamaterialAntennas/NewDir/SameModel")

  • To export the model, run one of the following commands (embed license is required):

evaluator.export_c("./MetamaterialAntennas/MetamaterialAntennas.h", precision="float")
evaluator.export_onnx("./MetamaterialAntennas/MetamaterialAntennas.onnx", precision="float")
evaluator.export_fmu("./MetamaterialAntennas/MetamaterialAntennas.fmu")
evaluator.export_vba("./MetamaterialAntennas/MetamaterialAntennas.bas")

Warning

Once the NeurEco object is no longer needed, free the memory by deleting the object by calling the delete method. For the example above, three objects must be deleted:

builder.delete()
evaluator.delete()
model.delete()