The synthetic data generation tool performs a series of steps:

Required preparations by the user:

The user shoulds prepare the following aspects to synthesize data:

• Dataset: Only categorical, numerical, or time data can be processed. Datasets may contain a maximum of 8 columns, must have a header with column names and do not require an index column.
• Method: By default, the CART method is used to generate synthetic data. CART generally produces higher quality synthetic data, but might not work well on datasets with categorical variables with 20+ categories. Use Gaussian Copula in those cases.
• Number of synthetic data points: Number of synthetic data points to be generated by the tool. Due to computational contstraints of browser-based synthetic data generation, the maximum is set to 5.000.

Performed by the tool

The following steps are performed by the tool:

Step 1. Data types detection:

The tool detects the type of data for each column of the attached dataset (numerical, categorical or datetime).

Step 2. Handling missing data:

• Missing at Random (MAR): The probability of data being missing is related to the observed data but not the missing data itself. The missingness can be predicted by other variables in the dataset. Example: Students’ test scores are missing, but the missingness is related to their attendance records. For this scenrario missing data is imputed.
• Missing Not at Random (MNAR): The probability of data being missing is related to the missing data itself. There is a systematic pattern to the missingness that is related to the unobserved data. Example: Patients with more severe symptoms are less likely to report their symptoms, leading to missing data that is related to the severity of the symptoms. For this scenrario missing data is imputed.
• Missing Completely at Random (MCAR): The probability of data being missing is completely independent of both observed and unobserved data. There is no systematic pattern to the missingness. Example: A survey respondent accidentally skips a question due to a printing error. In this scenario missing data is removed.

More information on the concepts MCAR, MAR en MNAR can be found in the book cby prof. Stef van Buuren, Utrecht University.

Step [unnumbered] Preprocessing:

Transforms all data into numerical format using LabelEncoder for less than 10 unique value per categorical column, OneHotEncoder for less than 50 unique value per categorical column and FrequencyEncoding for other cases. StandardScaler is used for numerical data.

Step 3. Synthesize method

• CART: The CART (Classification and Regression Trees) method generates synthetic data by learning patterns from real data through a decision tree that splits data into homogeneous groups based on feature values. It predicts averages for numerical data and assigns the most common category for categorical data, using these predictions to create new synthetic points.
• Gaussian Copula: Gaussian Copula works in two main steps: 1. The real data is transformed into a uniform distribution. Correlations between variables are modeled using a multivariate normal distribution (the Gaussian copula); and 2. Synthetic data is created by sampling from this Gaussian copula and transforming the samples back to the original data distributions.

Step [unnumbered] Postprocessing:

Encoded data are transformed back to it’s original format.

Step 4. Evaluation:

The generated synthetic data are evaluated in various ways.

Step 4.1 Visualization:

Univariate and bivariate plots are created to compare the generated synthetic data with the real data. For comparing categorical variables, bar charts are plotted. For comparing a numerical and a categorical variables, a violin plot is created. For comparing numercial variables, a LOESS plot is shown. For all plots holds: the synthetic data is of high quality when the shape of the distributions in the synthetic data equal the distributions in the real data.

Step 4.2 Diagnostic report:

For each column, diagnostic results are computed for the quality of the generated synthetic data. The computed metrics depend on the type of data.

Diagnostic results:

For numerical (or datetime) columns the following metrics are computed:

• Missing value similarity: Compares whether the synthetic data has the same proportion of missing values as the real data for a given column;
• Range coverage: Measures whether a synthetic column covers the full range of values that are present in a real column;
• Boundary adherence: Measures whether a synthetic column respects the minimum and maximum values of the real column. It returns the percentage of synthetic rows that adhere to the real boundaries;
• Statistic similarity: Measures the similarity between real column and a synthetic column by comparing the mean, standard deviation and median;
• Kolmogorov–Smirnov (KS) complement: Measures the maximum difference between the cumulative distribution functions (CDFs) of numerical columns in the synthetic and real dataset.

For categorical columns the following metrics are computed:

• Missing value similarity: Compares whether the synthetic data has the same proportion of missing values as the real data for a given column;
• Category coverage: Measures whether a synthetic column covers all the possible categories that are present in a real column;
• Category adherence: Measures whether a synthetic column adheres to the same category values as the real data;
• Total variation (TV) complement: Measures the maximum difference between the cumulative distribution functions (CDFs) of categorical columns in the synthetic and real dataset.

💯 For high-quality synthetic data, all values should be close to 1.0, but at least higher than 0.85.

Correlation matrix:

The one-to-one correlations between variables in the synthetic and real data are calculated, indicating the strength and direction of their linear relationships. The correlation matrix of the generated synthetic and real data should roughly display the same patterns.

Efficacy metrics:

cComparing real and synthetic datasets for downstream predictive tasks, such as regression and classification. The idea is to train a predictive model on synthetic data and evaluate its performance on real data.

For regression (when the target variable is numerical), the following metrics are computed:

• Mean squared error (MSE): average squared difference between predicted and actual values, quantifying the accuracy of a model’s predictions by penalizing larger errors more heavily;
• Mean absolute error (MAE): average magnitude of the errors between predicted and actual values, providing a straightforward assessment of model accuracy without emphasizing large errors;
• R² score: quantifies how well a model’s predictions match the actual data by measuring the proportion of variance in the target variable explained by the model.

For classification (when the target variable is categorical), the following metrics are computed:

• Accuracy score: measures the proportion of correctly predicted instances out of the total instances, providing an overall assessment of a model’s performance in classification tasks;
• Weighted F1 score: harmonic mean of precision and recall, calculated for each class and weighted by the class’s support (number of true instances), providing a balanced performance measure for imbalanced datasets.

Privacy metrics:

Computing the disclosure protection metric for synthetic data. This metric measures the proportion of synthetic records that are too similar (within a defined threshold) to real records, posing a disclosure risk.

Step 5. Download:

The generated synthetic data can de downloaded as csv and as json file. Evaluation of the synthetic data according to the above metrics can be downloaded as a evaluation report in pdf.

Documentation

More documentation about the tool and underlying SDG methods can be found on Github.