Machine Learning Glossary#

Michael J. Pyrcz, Professor, The University of Texas at Austin

Twitter | GitHub | Website | GoogleScholar | Geostatistics Book | YouTube | Applied Geostats in Python e-book | Applied Machine Learning in Python e-book | LinkedIn

Chapter of e-book “Applied Machine Learning in Python: a Hands-on Guide with Code”.

Cite this e-Book as:

Pyrcz, M.J., 2024, Applied Machine Learning in Python: A Hands-on Guide with Code [e-book]. Zenodo. doi:10.5281/zenodo.15169139 DOI

The workflows in this book and more are available here:

Cite the MachineLearningDemos GitHub Repository as:

Pyrcz, M.J., 2024, MachineLearningDemos: Python Machine Learning Demonstration Workflows Repository (0.0.3) [Software]. Zenodo. DOI: 10.5281/zenodo.13835312. GitHub repository: GeostatsGuy/MachineLearningDemos DOI

By Michael J. Pyrcz
© Copyright 2024.

This chapter is a summary of essential Machine Learning Terminology.

Motivation for this Glossary#

Firstly, why create this glossary?

I received a request for a course glossary from students in my Subsurface Machine Learning graduate course. While I usually dedicate a definition slide in my lecture slide decks to important terms, various students requested a consolidated glossary of terminology as part of their course review materials. The e-book provides an excellent vehicle and motivation for creating this resource.

Let me begin with a confession. There is a Machine Learning Glossary written by Google developers. For those seeking the in depth, comprehensive list of machine learning terms please use this book! For those seeking a comprehensive and in-depth reference of geostatistical terminology, this book remains an excellent resource.

So why create another glossary?

  • Scope - By writing my own glossary, I can limit the scope and descriptions to the concepts covered in this course. I believe many students would be overwhelmed by the size, breadth, and mathematical notation of a comprehensive geostatistics glossary.

  • Statistical Linkages - At the same time in my opinion machine learning is statistical learning and I have taken liberty to include many standard statistical terms as a foundation for all things machine learing.

  • Workflow Integration - By integrating the glossary directly into the e-book, I can link terminology to detailed chapter discussions, demonstrations, and examples. The goal is to eventually populate all chapters with hyperlinks to glossary entries, enabling students to move easily between concepts and applications.

  • Evergreen Resource - Finally, like the rest of this e-book, I want the glossary to be an evergreen living document. It will continue to evolve with new concepts, improved explanations, and feedback from students and readers.

I put quite a bit of time into this project during summer 2026 and I am happy with the way that is has evolved,

  • More than a glossary, it has become an evergreen network of machine learning concepts.

Sample#

A subset of values and locations measured from a population and used to infer parameter(s) of the population.

Examples include,

  • sparse spatial samples - 1,000 porosity measures from well log data in a reservoir with high measurement precision, but very small volume support and limited spatial coverage.

  • dense spatial samples - 1,000,000 acoustic impedance measurements over a 1,000 x 1,000 2D grid for a reservoir unit of interest with lower measurement precision and larger volume support.

In spatial modeling, the information content of a sample depends not only on the number of measurements, but also on the spatial distribution, measurement precision, and volume support.

  • also the amount of information in the sample set may be related to data locations and spatial continuity, i.e., the degree of redundancy between the samples.

The above states, sample is the collection of extracted values or observation, but there are two other important uses of the term sample,

  • verb - the act of extracting one or more values from a population, data set, or model, i.e., collecting data or realizations

  • singular noun - a single extracted value or observation

Used in:

Contrast with:

Scatter Plot#

A common data visualization plot that displays paired observations of two features as points to visualize their relationship, dependence, trends, clusters, and outliers.

For paired samples of two features, each observation is represented as a point,

\[ (x_\alpha, y_\alpha), \quad \alpha = 1,\ldots,n \]

where one feature is plotted on the x-axis and the other feature is plotted on the y-axis.

Scatter plots are used to visually assess,

  • relationship and dependence between features

  • trends and nonlinear patterns

  • clusters and populations within the data

  • outliers and anomalous observations

Note that association observed in a scatter plot does not necessarily imply causation.

Used in:

Compare with:

Scenarios#

Multiple subsurface models calculated by changing input parameters, assumptions, or modeling choices to represent uncertainty due to incomplete knowledge of the system.

Examples of scenario uncertainty include,

  • changing the input feature distributions, for example, modeling low, mid, and high porosity mean scenarios and generating subsurface models from each distribution

  • changing geological interpretations, such as alternative facies proportions, structural interpretations, or depositional models

  • changing model parameters, such as variogram parameters, trend models, or spatial continuity assumptions

Each scenario may include an ensemble of realizations generated by varying the random number seed in stochastic simulation to represent spatial uncertainty within that scenario.

Used in:

Contrast with:

Secondary Data#

Data samples of a feature other than the feature being modeled, used to improve estimation or simulation of the primary feature.

Secondary data are integrated through a model of the relationship between the secondary and primary data.

For example,

  • acoustic impedance measurements from seismic data (secondary data) are used to support calculation of a 3D porosity model, where porosity is the feature of interest

  • porosity measurements (secondary data) are used to support calculation of a permeability model, where permeability is the feature of interest

Secondary data may provide additional spatial information, trends, or constraints, but are not the direct observations of the feature being modeled.

Used in:

  • TBD

Contrast with:

Seismic#

A geophysical measurement technique that uses controlled acoustic sources and receivers to measure subsurface reflections and infer geological structure and rock properties.

Reflection seismic data provide high spatial coverage but generally lower resolution compared with direct measurements such as well log and core data.

Some important details include,

  • seismic reflection amplitudes are processed and inverted to estimate rock properties, such as acoustic impedance, calibrated and positionally aligned with well sonic logs

  • seismic provides a geological framework by identifying bounding surfaces, structural features, and reservoir extents

  • seismic provides soft information for reservoir properties, such as porosity and facies, through relationships established between seismic attributes and available primary data

In geostatistical modeling, seismic is commonly used as secondary data to improve spatial prediction and uncertainty models.

Used in:

  • TBD

Shapley Value#

A local measure of feature importance from explainable machine learning, borrowed from cooperative game theory. In cooperative game theory, Shapley values are used to,

  • partition the winnings among players

  • assign the contribution of each player to the outcome of the game

The motivation for Shapley values is that many predictive machine learning models are highly accurate but difficult to interpret. Two general approaches are available to improve interpretability,

  1. reduce model complexity

  2. apply model-agnostic interpretation methods, such as Shapley values

To apply Shapley values to predictive machine learning,

For an individual prediction, the Shapley value of each predictor feature is calculated as its average marginal contribution over all possible combinations (coalitions) of the remaining predictor features.

The prediction is decomposed into,

\[ \hat{y}=\bar{y}+\sum_{j=1}^{m}\phi_j \]

where \(\bar{y}\) is the average response feature value and \(\phi_j\) is the Shapley value for predictor feature \(j\).

Therefore,

  • positive Shapley values push the prediction above the response feature average

  • negative Shapley values pull the prediction below the response feature average

  • the Shapley values sum exactly to the prediction minus the response feature average

For feature ranking, a global measure of feature importance is obtained by averaging the magnitude (absolute value) of the Shapley values over many predictions,

  • larger average absolute Shapley values indicate greater overall feature importance

Shapley value summary,

  • model agnostic - applicable to virtually any predictive machine learning model

  • local explanation - explains an individual prediction

  • global feature importance - obtained by summarizing local Shapley values over many predictions

  • units are the same as the response feature

In addition to model explainability, Shapley values are used for,

Used in:

Also see:

Simpson’s Paradox#

A statistical phenomenon where a trend observed within individual groups reverses or disappears when the groups are combined into a single dataset.

For example,

  • each group may have a negative correlation between two features, while the combined dataset has a positive correlation

  • conversely, each group may show a positive relationship while the combined dataset shows a negative relationship

Simpson’s paradox occurs because,

  • a confounding feature influences both the grouping of the data and the relationship between the variables of interest

  • aggregating over the confounding feature changes the weighting of observations and can produce a misleading overall trend

As a result,

  • exploratory data analysis should examine both the complete dataset and meaningful subgroups

  • apparent relationships in aggregated data should be interpreted with caution when confounding variables may be present

Used in:

Also see:

Simulation#

A stochastic process of obtaining one or more possible values of a feature at unsampled locations that are consistent with available data and a multivariate, temporal and spatial uncertainty model.

Simulation models are designed to reproduce global characteristics and spatial variability, known as global accuracy, where the model reproduces specified global measures, including,

Unlike estimation methods, simulation produces multiple equiprobable realizations that represent uncertainty rather than a single optimal prediction.

Examples of simulation models include,

  • geostatistical subsurface heterogeneity models, including sequential Gaussian simulation, sequential indicator simulation, multiple point simulation, and object-based simulation

  • uncertainty propagation through a transfer function, including Monte Carlo simulation

Use simulation when,

  • reproducing feature distributions is important, especially when extreme values influence decisions

  • realistic spatial models are required for applications such as flow simulation

  • uncertainty in the decision criteria must be quantified through multiple possible models

Used in:

Contrast with:

Soft Data#

Data with significant uncertainty such that the information is represented probabilistically and uncertainty must be integrated into the spatial model.

For example,

Soft data integration requires workflows that incorporate uncertainty in the conditioning information, such as,

  • indicator kriging

  • sequential indicator simulation

  • p-field simulation

  • workflows that randomize or transform soft information into data realizations compatible with simulation methods that traditionally assume hard data, such as sequential Gaussian simulation

Soft data integration is an advanced topic and an active area of research; however, many standard subsurface modeling workflows and commercial software packages include approaches for incorporating soft information.

Used in:

  • TBD

Contrast with:

Spatial Estimation#

The process of obtaining a single best value to represent a feature at an unsampled location or time, \(\bf{u}\).

Given spatial data, \(z(\bf{u}_1), \dots, z(\bf{u}_n)\), we estimate the unknown feature value at location \(\bf{u}\) with a linear combination of the available data,

\[ z^{*}(\bf{u}) = \sum_{\alpha=1}^{n} \lambda_{\alpha} z(\bf{u}_{\alpha}) \]

An unbiasedness constraint may be added by assigning the remainder of the weight (one minus the sum of weights) to the global average. Therefore, if no informative data are available, the estimate approaches the global average of the feature,

\[ z^{*}(\bf{u}) = \sum_{\alpha=1}^{n} \lambda_{\alpha} z(\bf{u}*{\alpha}) + \left(1-\sum*{\alpha=1}^{n}\lambda_{\alpha}\right)\overline{z} \]

Some additional concepts,

  • local accuracy takes precedence over global accuracy, meaning that spatial estimation methods prioritize matching nearby observations over reproducing global statistics such as the histogram and variogram

  • spatial estimation maps and models generally have reduced variance and increased spatial continuity, resulting in smoother models than the true heterogeneous feature distribution

  • estimation models are not appropriate for transfer functions that are sensitive to heterogeneity and feature distributions, such as flow simulation or economic optimization

  • spatial estimation produces a single deterministic model and therefore does not provide multiple realizations required to sample uncertainty in the decision criteria; simulation methods are required for comprehensive uncertainty modeling and decision support

Examples of spatial estimation methods include,

There are also general non-spatial estimation methods; for example, many predictive machine learning models perform estimation by focusing on local predictive accuracy rather than global distribution reproduction,

Contrast with:

Spatial Sample Selection#

The process of selecting locations for collecting subsurface samples to reduce uncertainty and support resource development decisions.

For subsurface resource exploration and development, sample locations are selected to achieve two primary objectives:

  1. Reduce uncertainty - by collecting information to answer key geological and engineering questions, for example,

  • how far does the contaminant plume extend? – sample the plume periphery to define its extent

  • where is the fault? – collect data guided by seismic interpretation and geological hypotheses

  • where are the highest mineral grades? – sample areas with potential economic significance

  • how far does the reservoir extend? – offset drilling to define reservoir boundaries

  1. Maximize net present value - by collecting information while advancing development objectives, for example,

  • maximize production rates

  • maximize recoverable resource or mineral tonnage

Therefore, subsurface samples are often collected for dual purposes: reducing uncertainty and supporting development. For example,

  • exploration and appraisal wells provide geological and reservoir information that can subsequently be incorporated into the production system

  • production wells provide operational data while also becoming valuable conditioning data for future reservoir models

Used in:

Spectral Clustering#

Spectral Clustering: a partitional clustering method that utilizes the spectrum, eigenvalues and eigenvectors, of a matrix that represents the pairwise relationships between the data.

Advantages of spectral clustering,

  • the ability to encode pairwise relationships, integrate expert knowledge.

  • eigenvalues provide useful information on the number of clusters, based on the degree of ‘cutting’ required to make k clusters

  • lower dimensional representation for the sample data pairwise relationships

  • the resulting eigenvalues and eigenvectors can be interpreted, eigenvalues describe the amount of connection for each number of groups and eigenvectors are grouped to form the clusters

Used in:

Standard Deviation#

The square root of the variance. Standard deviation measures the spread of a feature about its average in the same units as the original feature.

Given the sample variance,

\[ s^2 = \frac{1}{n-1}\sum_{\alpha=1}^{n}\left(x_{\alpha}-\overline{x}\right)^2 \]

the sample standard deviation is,

\[ s = \sqrt{s^2} \]

The equivalent population parameters are, the population variance,

\[ \sigma^2 = \frac{1}{n}\sum_{\alpha=1}^{n}\left(x_{\alpha}-\overline{x}\right)^2 \]

the population standard deviation is,

\[ \sigma = \sqrt{\sigma^2} \]

Used in:

  • TBD

Also see:

Standardization#

Distribution rescaling that can be thought of as shifting, and stretching or squeezing of a univariate distribution (e.g., histogram) to a mean of 0.0 and a standard deviation of 1.0.

  • a special case of an affine transformation

For each data value, \(x_i\), the following transformation maps it to the standardized value, \(y_i\),

\[ y_i=\frac{x_i-\overline{x}}{\sigma_x}, \quad \forall \quad i=1,\ldots,n \]

where \(\overline{x}\) is the original mean and \(\sigma_x\) is the original standard deviation.

This transformation is linear and therefore,

  • preserves the rank ordering of observations

  • does not change the shape of the distribution

  • changes only the location and scale of the feature values

Standardization is useful when feature magnitude influences model behavior, for example,

Standardization is commonly applied to predictor features and may also be applied to response features. Predictions are then back-transformed to the original units for interpretation.

Used in:

Also see:

Stationarity#

The decision that a subset of the subsurface is the same “stuff” and therefore can be pooled to calculate statistics and build models.

Replicates are required to calculate any statistic. In many applications, replicates are obtained by repeated measurements through time, for example,

  • air or water samples collected repeatedly from a monitoring station

For subsurface resource models,

  • repeated samples are generally not available at the same location; only one sample is available at each location

  • instead of pooling measurements through time, we must pool samples over space to calculate statistics

Why must we pool data? Ultimately, it is required to make inference about the population from a limited sample,

  • to calculate statistics

  • to build spatial models

The choice of stationary domain is an expert geological decision. Without a stationarity decision, we are restricted to the measured locations (well bores or drill holes) and cannot calculate statistics or make predictions between samples.

An example geological definition of stationarity could be:

The rock within the stationary domain is sourced, deposited, preserved, and post-depositionally altered in a similar manner. The domain is mappable and may be used for local prediction or as information for analogous locations within the subsurface; therefore, information may be pooled over this expert-defined volume of the subsurface.

This expert geological interpretation defines a domain over which statistical stationarity is assumed for modeling.

There are two aspects of any stationarity decision:

  1. Import license - the choice of which samples are allowed to contribute to the calculation of a statistic

  2. Export license - the choice of where the resulting statistic is applicable within the subsurface

To state a stationarity decision, we must specify:

  1. the statistic assumed stationary, for example, the mean, variance, cumulative distribution function , or spatial continuity

  2. the spatial domain over which the statistic is assumed stationary, for example, the entire model, a facies, a depositional environment, or a geological region

Examples of statistical definitions of stationarity include:

  • stationary mean

\[ E[Z(\mathbf{u})] = \overline{z}, \quad \forall \mathbf{u} \in AOI \]

  • stationary cumulative distribution function

\[ F_z(\mathbf{u},z) = F_z(z), \quad \forall \mathbf{u} \in AOI \]

  • stationary semivariogram

\[ \gamma_z(\mathbf{u},\mathbf{h}) = \gamma_z(\mathbf{h}) \]

The stationarity decision may be extended to any statistic of interest, including,

Additional considerations for stationarity include:

  • Stationarity is a decision, not a hypothesis - therefore, it is not directly tested. Instead, data may demonstrate that a chosen stationarity decision is inappropriate.

  • Stationarity depends on scale - the appropriate modeling scale should be selected based on the geological process, decision objective, and project requirements.

  • A stationarity decision cannot be avoided - without stationarity, spatial statistics cannot be calculated and modeling cannot progress beyond measured locations. Conversely, assuming broad stationarity over very large regions of the Earth is generally unrealistic.

  • Geomodeling stationarity is a domain decision - defining (1) where data may be pooled (import license) and (2) where resulting statistics may be applied (export license).

  • Nonstationary trends may be modeled explicitly - deterministic trends can be removed and the remaining stationary residual variation can be modeled stochastically. This is the hybrid modeling approach.

Statistic#

A function of sample data that summarizes a property of the sample. Examples include,

Statistics are calculated from available samples because the complete population is generally unknown.

How do we use statistics?

Used in:

Compare with:

Statistical Distribution#

A description of the frequency or probability behavior of a feature over the range of possible values.

A univariate statistical distribution describes how feature values are distributed without considering their spatial or temporal arrangement. We represent the statistical distribution with,

What do we learn from a statistical distribution? For example,

  • what are the minimum and maximum values?

  • what is the most common range of values?

  • do we have many low values?

  • do we have many high values?

  • are there outliers or values that do not make geological or physical sense and require explanation?

  • what is the variability and uncertainty in the feature values?

Statistical distributions are fundamental for inference, simulation, and uncertainty modeling.

Used in:

Statistics#

The theory and practice for collecting, organizing, and interpreting data, as well as drawing conclusions and making decisions.

Used in: Entire book

Same as:

Stochastic Gradient-based Optimization#

Optimization method commonly applied in machine learning to estimate model parameters by iteratively minimizing a loss function.

Compared with full gradient-based optimization, stochastic gradient-based optimization improves computational efficiency by calculating gradients from random batches of the training data rather than the complete dataset.

  • a batch is a random subset of the training data with size \(n_{batch}\)

  • the batch provides a stochastic approximation of the full loss function gradient

  • each optimization step is less accurate than using the complete dataset, but much faster to calculate

  • increasing \(n_{batch}\) improves gradient accuracy while decreasing stochasticity

  • decreasing \(n_{batch}\) increases stochasticity and reduces computational cost per optimization step

The general workflow is,

  1. initialize the model parameters, \(\mathbf{b}\)

  2. randomly select a batch of training data

  3. calculate the loss function and its gradient over the batch,

\[ \nabla_{\mathbf b}L(\mathbf b) \]

  1. update the model parameters,

\[ \mathbf b_{t+1}=\mathbf b_t-r\nabla_{\mathbf b}L(\mathbf b_t) \]

where \(r\) is the learning rate (step size).

This process is repeated until a stopping criterion is satisfied, such as,

  • maximum number of optimization iterations

  • sufficiently small change in the loss function

  • sufficiently small parameter updates

Common extentions include,

  • momentum - to dampen oscillation and between track large scale gradients in the loss function

  • adaptive learning rates - with methods such as AdaGrad, RMSProp, and Adam

Stochastic Model#

A model of a system or process that includes uncertainty and is represented by multiple possible outcomes, including realizations and scenarios, constrained by available data, statistics, and modeling assumptions.

Stochastic models represent uncertainty by describing a range of plausible outcomes rather than a single deterministic prediction.

Examples include,

  • data-driven models that integrate uncertainty, such as geostatistical simulation models

  • Monte Carlo models that propagate uncertainty through a transfer function

  • ensemble machine learning models that represent prediction uncertainty

Advantages:

  • computational speed compared with many physics-based models

  • explicit uncertainty assessment

  • ability to report confidence Intervals, prediction intervals, and risk measures

  • ability to integrate many sources of data and information

  • flexible data-driven approaches

Disadvantages:

  • limited representation of underlying physics unless explicitly incorporated

  • dependence on statistical model assumptions and simplifications

  • uncertainty models may be incomplete if important processes or information are not represented

Contrast with:

Stratified K-fold Cross Validation#

A K-fold Cross Validation-based cross validation method that preserves the class proportions of the response feature within each fold.

Stratification provides more representative training and testing subsets for classification problems, particularly when classes are imbalanced.

Contrast with:

Subsurface Modeling Workflow#

A common geostatistical workflow for integrating subsurface data, modeling uncertainty, and supporting development decision making. The workflow proceeds from data to decisions through the following steps:

  1. Integrate all available information to build multiple subsurface scenarios and realizations that sample the uncertainty space.

  2. Apply all realizations through the transfer function to sample the uncertainty in the decision criteria.

  3. Assemble the distribution of the decision criteria from the ensemble of realizations and scenarios.

  4. Make the optimum reservoir development decisions while accounting for the modeled uncertainty.

Supervised Learning#

Machine learning methods that learn relationships between predictor features and labeled response features.

Supervised learning uses both predictor features and response features,

  • response feature values, \(Y\), are provided with corresponding predictor features, \(X_1,\ldots,X_m\)

  • the machine learns a mapping from predictor features to response features

  • learned relationships may include prediction functions, decision boundaries, probability models, or other representations of the relationship between inputs and outputs

Supervised learning focuses on prediction of a response feature rather than inference of the natural system,

  • estimating response feature values for new observations

  • evaluating model performance using unseen data

Common supervised learning methods include,

  • regression - predicting continuous response features

  • classification - predicting categorical response features

  • time series forecasting - predicting future response feature values from historical observations

In this course we use the terms,

  • predictive machine learning - supervised learning methods focused on predicting response features from predictor features

  • inferential machine learning - unsupervised learning methods focused on discovering patterns and structure in data

Contrast with:

Support Vector#

For a support vector machine, support vectors are the training observations that determine the location of the decision boundary.

Support vectors include observations that,

  • lie on or inside the margin

  • are misclassified (soft-margin SVM)

Training observations well outside the margin,

  • have no influence on the fitted decision boundary

Only the support vectors contribute to the optimization of the support vector machine, giving the method its name.

Used in:

Also see:

Support Vector Machine#

Predictive, binary classification machine learning method designed to perform well when categorical groups have poor separation in the original predictor feature space.

Conceptually, support vector machines project the original predictor features into a higher-dimensional feature space where a linear decision boundary (a plane or hyperplane) can separate the categories,

\[ f(x)=x^T\beta+\beta_0 \]

where \(\beta\) is the vector of model parameters, \(\beta_0\) is the intercept, and \(x\) is the predictor feature vector in the higher-dimensional feature space.

The predicted category is determined by,

\[ G(x)=\operatorname{sign}(f(x)) \]

where,

  • \(f(x)\) is proportional to the signed distance from the decision boundary

  • \(f(x)=0\) lies exactly on the decision boundary

  • \(G(x)=-1\) and \(G(x)=+1\) indicate the two classification categories

The kernel trick makes this practical,

  • rather than explicitly constructing the higher-dimensional feature space, support vector machines calculate inner products between observations using a kernel function

  • the classifier therefore behaves as though it operates in the higher-dimensional feature space while avoiding the associated computational cost

For perfectly separable data, the decision boundary must satisfy,

\[ y_i\left(x_i^T\beta+\beta_0\right)\ge1 \]

where the class labels are encoded as \(y_i\in\{-1,+1\}\).

For real-world problems with overlapping groups and noisy observations, a soft-margin support vector machine allows margin violations through slack variables,

\[ y_i\left(x_i^T\beta+\beta_0\right)\ge1-\xi_i \]

where,

  • \(\xi_i=0\) indicates an observation outside the margin and correctly classified

  • \(0<\xi_i<1\) indicates an observation inside the margin but correctly classified

  • \(\xi_i>1\) indicates a misclassified observation

The optimization problem is,

\[ \underset{\beta,\beta_0}{\min}\left(\frac{1}{2}\|\beta\|^2+C\sum_{i=1}^{n}\xi_i\right) \]

subject to,

\[ \xi_i\ge0,\qquad y_i(x_i^T\beta+\beta_0)\ge1-\xi_i \]

The optimization balances two competing objectives,

  • maximize the decision margin

  • minimize margin violations through the slack variables

Only a subset of the training data influences the fitted classifier,

  • observations on or within the margin are known as support vectors

  • observations well outside the margin have no influence on the fitted decision boundary

The hyperparameter \(C\) controls the model bias–variance trade-off,

  • larger \(C\) places greater emphasis on correct classification, resulting in a smaller margin and potentially higher model variance (overfitting)

  • smaller \(C\) allows more margin violations, resulting in a wider margin and potentially higher model bias (underfitting)

Support vector machines become nonlinear through the choice of kernel function. Common kernels include,

  • Linear kernel

  • Polynomial kernel

  • Radial basis function (RBF) kernel

  • Sigmoid kernel

Additional kernel hyperparameters control the flexibility of the classifier, for example,

  • Polynomial kernel - polynomial order

  • Radial basis function kernel - \(\gamma\), controlling the distance over which training observations influence the decision boundary

Support vector machine summary,

  • effective for high-dimensional predictor feature spaces

  • depends only on the support vectors near the decision boundary

  • maximizes the decision margin while penalizing margin violations

  • uses the kernel trick to efficiently construct nonlinear classifiers

Used in:

Also see:

Tabular Data#

A data representation where observations are organized into a table with,

Tabular data is the most common data format for machine learning, statistics, and data analytics. Examples include,

  • spatial data with one row for each sampled location

  • temporal data with one row for each time observation

  • multivariate laboratory measurements with one row for each sample

Pandas’ ‘DataFrame’ is the standard Python class for working with tabular data due to,

  • convenient storage, access, and manipulation of tabular data

  • built-in methods to load data from a variety of file formats, databases, and spreadsheets

  • built-in methods for summary statistics, visualization, grouping, filtering, sorting, and joining tables

  • built-in methods for cleaning, reshaping, and transforming data

  • built-in attributes describing the data structure, for example, dimensions, column names, data types, and missing values

Compare with:

Also see:

Testing Data#

The subset of a dataset reserved for evaluating the predictive performance of a trained machine learning model on previously unseen data.

Also see:

Train and Test Split#

Model cross validation, prior to predictive model training, withholds a proportion of the data as testing data.

Model hyperparameter tuning selects the combination that minimizes the error norm over the withheld testing data.

The most common approach is random selection; however, this may not provide fair testing.

The testing difficulty should be similar to the intended real-world use of the model,

  • too easy – testing cases are the same as, or very similar to, training cases. Random sampling is often too easy.

  • too hard – testing cases are very different from the training cases, requiring severe extrapolation beyond the available data.

Cross validation may use a single train and test split or multiple splits, for example,

Used in:

Training Data#

The subset of a dataset used to train a machine learning model by estimating its model parameters through model parameter training.

Also see:

Training Image#

A 2D training image is a dense conceptual representation of expected geological patterns, connectivity, and morphology over a 2D area of interest.

Training images provide a library of geological patterns used to inform,

  • geostatistical multiple point simulation, where complex spatial relationships are learned from examples rather than only two-point statistics.

  • genAI models

Common aspects of training images include,

  • represent prior geological knowledge and conceptual understanding of spatial patterns

  • do not include local information or conditioning data before simulation conditioning

  • must have the same cell size as the simulation model

  • do not need to have the same extent (number of model cells in each dimension), but should be large enough to provide sufficient examples of geological patterns

  • larger training images provide more pattern examples and greater representation of geological variability, but increase computational complexity

2D training images and 3D training models are used as conceptual pattern libraries for machine learning, generative AI models, and geostatistics,

  • generative models learn the statistical structure of patterns from training images and models are used as conceptual pattern libraries for machine learning, generative AI models, and geostatistics,

  • generative models learn the statistical structure of patterns from training images and models

  • generative models learn the statistical structure of patterns from training images

  • generated realizations reproduce learned spatial relationships while creating new possible outcomes

Used in:

Also see:

Training Model#

A 3D training model is a dense conceptual representation of expected geological patterns, connectivity, and morphology over a 3D volume of interest.

Training models provide a library of geological patterns used to inform,

  • geostatistical multiple point simulation, where complex spatial relationships are learned from examples rather than only two-point statistics.

  • genAI models

Common aspects of training models include,

  • represent prior geological knowledge and conceptual understanding of spatial patterns

  • do not include local information or conditioning data before simulation conditioning

  • must have the same cell size as the simulation model

  • do not need to have the same extent (number of model cells in each dimension), but should be large enough to provide sufficient examples of geological patterns

  • larger training models provide more pattern examples and greater representation of geological variability, but increase computational complexity

2D Training images and 3D training models are used as conceptual pattern libraries for machine learning, generative AI models, and geostatistics,

  • generative models learn the statistical structure of patterns from training images and models

  • generated realizations reproduce learned spatial relationships while creating new possible outcomes

Used in:

Also see:

Transfer Function#

A model, process, or calculation applied to spatial subsurface model realizations and scenarios to transform uncertain subsurface properties into a decision criteria.

The transfer function connects subsurface uncertainty models to decision making by calculating metrics that represent value, risk, health, environment, safety, or operational constraints.

Transfer functions may be physics-based, data-driven, or hybrid. Example transfer functions include,

  • transport and bioattenuation - numerical simulation to model soil contaminant concentrations over time during a pump-and-treat operation

  • volumetric calculation - estimate total oil-in-place from reservoir property models

  • heterogeneity metrics - calculate indicators related to recovery factor and estimate reserves from resources

  • flow simulation - generate pre-drill production forecasts for a planned well

  • Whittle pit optimization - calculate mineral resources and ultimate pit shell designs

Transformer#

A neural network architecture that uses attention mechanisms to learn relationships between elements of sequential or structured data.

Transformers learn contextual representations by:

  • computing attention-based relationships between input elements

  • combining information from multiple elements without requiring sequential processing

  • learning compact feature representations through stacked attention and neural network layers

Transformers are widely used in natural language processing, computer vision, and other machine learning applications.

Common transformer-based models include:

  • large language models (LLMs) - attention to learn large-scale, contextual relationships among language elements

  • vision transformers (ViTs) - attention to learn large-scale spatial structures in images

  • multimodal foundation models - attention to simultaneously process diverse data types to address diverse problems

Contrast with:

Trend#

An interpretation that a spatial feature is nonstationary over space, meaning that one or more statistics of the feature systematically change over the 2D area of interest or 3D volume of interest.

For example,

  • porosity decreases with depth

  • copper grade increases toward a highly faulted zone

Trend in spatial data may be identified by,

  • integrating expert geological knowledge and physical understanding

  • calculating bivariate statistics, such as conditional means given a spatial coordinate

  • fitting a deterministic non-stationarity model with location as a predictor feature and evaluating model significance

  • calculating an experimental variogram and checking for trend structure

Trend is also used to describe a deterministic model of nonstationarity in a statistic or metric of interest, as in Trend Model.

Used in:

Also see:

Trend and Residual Workflow#

Most geostatistical modeling methods assume stationarity in the feature mean. Yet, nonstationarity, trend, in the mean is commonly observed in the subsurface.

  • to address this limitation, the common hybrid workflow is to deconvolve the spatial data into 2 components:

  1. known - deterministic trend model

  2. unknown - stochastic residual model

The known trend is calculated and then subtracted from the data, leaving a residual that is modelled stochastically with uncertainty (treated as unknown). The following steps are applied:

  1. model the nonstationary, spatial, deterministic trend for a feature of interest

  2. subtract the trend from the data to calculate the residual

  3. model the residual with geostatistical spatial estimation or simulation

  4. add the deterministic trend to the geostatistical (deterministic if kriging or stochastic if simulation) residual

  5. check the model

Also see:

Trend Model#

A determistic model representing the spatial trend in a statistic that is applied as an input for a spatial simulation method, for example,

  • a linear function for reduction in average porosity with depth, based on local data and regional compaction trends

  • a moving window local average copper grade model to model the increase in copper grade toward the highly faulted zone

This provides a local value of the statistic at all model grid cells, so the simulation can apply the trend model to relax the assumption of statistionarity in the statistic.

  • a trend model may be calculated and applied to applied to any statistic used in the simulation model, e.g., mean, variogram range, variogram major direction, correlation coefficient, etc.

Also see:

Uncertainty Modeling#

Characterization of the range of plausible values for a feature at a location, jointly over the entire subsurface model, or propagated through a transfer function to support decision making.

Uncertainty may be considered at different levels:

Common sources of uncertainty include:

  1. Data imprecision - measurement error, interpretation uncertainty, and imperfect observations

  2. Spatial offset from data - uncertainty from estimating unsampled locations away from available spatial data

  3. Model parameter inference - uncertainty in inferred parameters such as global mean, variance, variogram, and correlation structure

  4. Conceptual model uncertainty - uncertainty from choices about geological framework, modeling approach, and assumptions about the subsurface system

Uncertainty models are represented with ensembles of scenarios and realizations:

  • Scenarios - multiple spatial subsurface models calculated by changing input parameters or other modeling choices to represent uncertainty from model parameters and conceptual choices

  • Realizations - multiple spatial subsurface models calculated by holding input parameters and modeling choices constant and changing only the random number seed

How can we address each source of uncertainty?

  • data imprecision - model data uncertainty through data realizations, soft data integration, or indicator transforms

  • spatial offset from data - calculate multiple stochastic realizations by varying the simulation random number seed

  • model parameter inference - calculate scenarios by varying inferred model parameters

  • conceptual model uncertainty - develop and compare alternative geological interpretations or modeling workflows

Important considerations for uncertainty modeling,

  • uncertainty modeling is critical for quantifying limitations in sample precision and model predictions

  • uncertainty is itself a model; there is no objective uncertainty independent of assumptions, data, and modeling choices. Failure to recognize this leads to the circular pursuit of “uncertainty in the uncertainty”

  • uncertainty results from sparse sampling, measurement error, interpretation uncertainty, bias, and geological heterogeneity

  • uncertainty reflects our limited ability to observe subsurface features with sufficient accuracy, resolution, and coverage; it is not an intrinsic property of the geology itself

Used in:

Underfit Model#

A predictive machine learning model that fails to learn the underlying relationship in the natural system due to insufficient flexibility or excessive constraints.

An underfit model demonstrates,

  • low prediction accuracy with training data and poor prediction accuracy with withheld testing data

  • insensitivity to the specific training observations, resulting in systematic prediction errors

During model hyperparameter tuning, the underfit region is characterized by,

  • increasing model complexity reduces both training error and testing error

  • strong agreement between training and testing performance, but both with high error

This behavior represents the high model bias region of the model bias–variance trade-off.

Issues associated with an underfit machine learning model include,

  • insufficient model complexity and flexibility relative to the available data quantity, accuracy, frequency, and coverage

  • failure to capture important patterns and relationships in the natural system

Underfitting may be reduced through,

  • increasing model complexity

  • reducing regularization or other constraints on model flexibility

  • including additional informative predictor features

More about underfit models,

  • underfit models often approach the global mean of the response feature

  • underfit models have high error over both training and testing data

  • increasing model complexity generally decreases error over the underfit region

  • the underfit region occurs before the optimal model complexity, where training and testing errors are both decreasing

Used in:

Contrast with:

Union of Events#

The union of events represents all outcomes where event \(A\) occurs, event \(B\) occurs, or both events occur. The probability of the union is calculated with the probability addition rule,

\[ P(A \cup B) = P(A) + P(B) - P(A \cap B) \]

The intersection probability is subtracted because outcomes where both \(A\) and \(B\) occur are included in both \(P(A)\) and \(P(B)\) and would otherwise be counted twice.

For mutually exclusive events, the intersection probability is zero,

\[ P(A \cap B)=0 \]

and the probability addition rule simplifies to,

\[ P(A \cup B)=P(A)+P(B) \]

Used in:

Univariate#

Involving a single feature or event only.

Examples include:

Used in:

Compare with:

Univariate Parameter#

A univariate population summary measure describing a single feature.

Examples include:

In practice, the complete population is rarely available, so univariate parameters are inferred from available univariate statistics calculated from samples.

Used in:

  • TBD

Contrast with:

Univariate Statistic#

A summary measure calculated from samples of a single feature.

Examples include:

Univariate statistics describe the available sample and are used to infer the corresponding ppopulation parameter.

Used in:

Contrast with:

Unsupervised Learning#

Machine learning methods that learn patterns, structure, or representations from data without labeled response features.

Unsupervised learning uses only predictor features,

  • no response feature, \(Y\), is provided; instead only predictor features, \(X_1,\ldots,X_m\), are available

  • the machine learns by discovering regularities, patterns, and compact representations of the data

  • learned structures may include feature projections, group assignments, latent neural network features, probability distributions, or other representations of the data

Unsupervised learning focuses on inference of the natural system rather than prediction of a response feature,

  • understanding the structure, variability, and relationships within the available data

  • identifying patterns that may support interpretation, exploration, or future modeling

Common unsupervised learning methods include,

  • clustering - identifying groups or regions of similar observations

  • dimensionality reduction - finding lower-dimensional representations that preserve important information

  • density estimation - modeling the distribution and probability structure of the data

In this course we use the terms,

  • inferential machine learning - unsupervised learning methods focused on discovering patterns and structure in data

  • predictive machine learning - supervised learning methods focused on predicting response features from predictor features

Contrast with:

Value#

The term value is used in 3 distinct cases:

  1. number representing a magnitude of a property

  2. measure of the benefit of a subsurface development, i.e., profit - loss, reduced environmental emissions, improved safety, reliable resources and energy

  3. component of the attention mechanism that allows a model to dynamically focus on the most relevant information while reducing the influence of less relevant information.

More on this last attention mechanism related definition,

  • A Value (\(V\)) is the stored information that may contribute to the current prediction or calculation.

The attention mechanism,

  • compares each Query with all Keys to calculate similarity scores, which are then used to derive attention weights.

  • the attention weights are applied to the Values and combined to provide the information used for the current prediction or calculation.

Some additional comments,

  • each Key–Value pair represents a stored piece of information, where the Key describes the information and the Value contains the associated content.

Conceptually,

  • Value (\(V\)) answers, “Here is the information to use.”

Used in:

Also see:

Variable#

Any property measured or observed in a study, for example,

  • porosity, permeability, mineral concentrations, saturations, contaminant concentration

  • in data mining / machine learning this is known as a feature

  • often requires significant analysis, interpretation, etc.

Used in:

Same as:

Variance#

A measure of distribution dispersion, the spread or variability of a feature about its average. Larger variance indicates greater variability.

For a sample, the variance is,

\[ s^2 = \frac{1}{n-1}\sum_{\alpha=1}^{n}\left(x_{\alpha}-\overline{x}\right)^2 \]

The equivalent population parameter is the population variance,

\[ \sigma^2 = \frac{1}{n}\sum_{\alpha=1}^{n}\left(x_{\alpha}-\overline{x}\right)^2 \]

Some comments about variance,

  • units - the units of variance are squares units of the feature, for more intuitive units consider using the standard deviation

  • additivity - variance are additive, enabling a lot of workflows like analysis of variance and trend + residual workflows, for example given \(X_{residual} + X_{trend} = X_{total}\), the variance is calculated as,

\[ \sigma^2_{X_{total}} = \sigma^2_{X_{trend}} + \sigma^2_{X_{residual}} + 2\,\mathrm{Cov}(X_{trend},X_{residual}) \]

  • momments - variance is the \(2^{nd}\) centered momment

  • outliers - variance is very sensitive to outliers

Used in:

Also see:

Variance Inflation Factor#

Feature ranking or often feature filtering metric based on the linear multicollinearity between a predictor feature (\(X_i\)) and all other predictor features (\(X_j, \forall j \ne i\)).

Variance inflation factor is often applied as a first-pass filter to remove highly redundant predictor features before subsequent feature ranking or model training.

To calculate variance inflation factor,

  1. Build a linear regression model to predict one predictor feature from all other predictor features,

\[ X_i = \sum_{j,j \ne i}^{m} b_j X_j + b_0 + \epsilon \]

  1. Determine the coefficient of determination, R-squared or \(R^2\), for this regression model,

  • this \(R^2\) represents how well the remaining predictor features explain the feature \(X_i\)

  1. Calculate the variance inflation factor,

\[ VIF_i = \frac{1}{1-R_i^2} \]

where \(R_i^2\) is the coefficient of determination from predicting \(X_i\) using all other predictor features.

The interpretation of variance inflation factor,

  • \(VIF=1\) - no linear redundancy with other predictor features

  • larger \(VIF\) values indicate increasing multicollinearity and reduced independent information from the predictor feature

  • large \(VIF\) values indicate that the uncertainty in estimated model coefficients is inflated due to redundant predictor features

Common guidelines include,

  • \(VIF < 5\) - often considered acceptable multicollinearity

  • \(VIF > 5\) or \(10\) - often considered evidence of problematic multicollinearity

Comments about variance inflation factor,

  • redundancy - accounts for linear relationships between a predictor feature and all other predictor features

  • relevance - does not account for any relationship between the predictor feature and the response feature

  • linearity - only identifies linear redundancy; nonlinear feature relationships may not be detected

Used in:

Also see:

Variance Reduction Factor#

A convenient factor used to correct variance when changing from the data volume support to a larger model volume support.

The variance reduction factor, \(f\), is defined as the ratio of variance at the larger volume support, \(v\), to the variance at the original data volume support, \(\cdot\):

\[ f = \frac{\sigma^2(v)}{\sigma^2(\cdot)} \]

Using volume-variance relations, this can be calculated as,

\[ f = 1 - \frac{\overline{\gamma}(v,v)}{\sigma^2} \]

where \(\overline{\gamma}(v,v)\) is the average variogram value within the volume support \(v\) and \(\sigma^2\) is the variance at the original data support.

Equivalently, using dispersion variance,

\[ f = \frac{D^2(v,V)}{D^2(\cdot,V)} = \frac{D^2(v,V)}{\sigma^2} \]

The variance reduction factor is applied to adjust the data histogram to represent the reduced variability expected at a larger model scale.

Without volume support correction, the original data-scale distribution will have excessive variance when applied directly to a larger model volume.

Used in:

Also see:

Variogram#

A scatterplot with axes of difference or variance over distance.

  • Experimental variogram is calculated over integer multiples of the unit lag distance and then plotted as points, then permissible variogram models are fit to the experimental variogram while integrating other domain and local knowledge.

  • the variogram is calculated as one half the average squared difference over lag distance, 𝐡, over all possible pairs of data,

\[ \gamma_z(\bf{h}) = \frac{1}{2 \cdot N(\bf{h})} \sum_{\alpha = 1}^{N(\bf{h})} \left( z(\bf{u}_{\alpha}) - z(\bf{u}_{\alpha} + \bf{h}) \right)^2 \]

  • the precise term is semivariogram (or variogram if you remove the \frac{1}{2} in the equation above), but in practice, the semivariogram is only used and the term variogram is always used for the semivariogram

  • the \(\frac{1}{2}\) term is added to the semivariogram so that the covariance function, \(C_z(\bf{u})\), and variogram, \(\gamma_z(\bf{h})\), may be related as:

\[ C_z(\bf{h}) = \sigma_z^2 - \gamma_z(\bf{h}) \]

Note the correlogram, \(\rho_z(\bf{u})\), is related to the covariance function, \(C_z(\bf{u})\), as:

\[ \rho_z(\bf{u}) = \frac{C_z(\bf{h})}{\sigma_z^2} \]

Here are some general observations about the variogram,

  1. Often increasing - as the lag Distance, \(\bf{h}\), increases, variability over the lag distance increase (in general).

  2. Not a local measure - the variogram is calculated with over all possible pairs separated by lag vector, \(\bf{h}\).

  3. Interpret relative to the [ill - we need to plot the sill on with the experimental variogram to know the degree of correlation.

  • the sill is the variance, \(\sigma^2\), given stationarity of the variance and variogram, \gamma_z(\bf{h})):

\[ C_z(\bf{h}) = \sigma_z^2 - \gamma_z(\bf{h}) \]

\(\quad\) and given a standardized feature, \(\sigma_z^2 = 1.0\),

\[ \rho_z(\bf{u}) = \sigma_z^2 - \gamma_z(\bf{h}) \]

\(\quad\) the distance from the sill to the experimental variogram is the correlation coefficient over the specific lag distance.

  1. Range - the lag distance at which the variogram reaches the sill is know as the range.

  • at the range, knowing the data value at the tail provides no information about a value at the head.

  1. Nugget effect - sometimes there is a discontinuity in the variogram at distances less than the minimum data spacing. This is known as nugget effect.

  • the ratio of nugget divided by sill, is known as relative nugget effect, reported in percentage, e.g., 10% relative nugget effect

  • we model the nugget effect as a no correlation structure over all lags greater than an infinitesimal distance, \(\bf{h} \gt \epsilon\)

  • measurement error, causes an apparent nugget effect, if this is suspected do not add nugget effect to the variogram model

Also see:

Venn Diagram#

A visual tool for communicating probability relationships using set notation and the probability of events.

A Venn diagram contains:

  • a box labelled as \(\Omega\) representing the sample space, including all possible outcomes

  • enclosed labelled shapes representing events, which are subsets of the sample space

What do we learn from a Venn diagram?

  • the size of regions is proportional to the probability of occurrence

  • the entire sample space, \(\Omega\), represents all possible outcomes and therefore has probability:

\[ P(\Omega)=1.0 \]

  • individual regions represent marginal probabilities, for example:

\[ P(A) \]

  • overlapping regions represent joint probabilities, for example:

\[ P(A \cap B)=P(A,B) \]

  • overlapping regions relative to a conditioning event represent conditional probabilities, for example:

\[ P(A|B)=\frac{P(A\cap B)}{P(B)} \]

Venn diagrams are an excellent tool to visualize marginal probability, joint probability, and conditional probability relationships and are especially useful for understanding probability operators.

Used in:

Volume of Interest#

The 3D spatial domain that is being characterized, modeled, and evaluated to support subsurface decision making. In general, the volume of interest,

  • is the subsurface reservoir for oil and gas, the ore body for mining, or the aquifer for hydrogeological applications

  • may include volume away from the reservoir or ore body to support data integration and extraction modeling

  • may be further subdivided into local regions or facies and modeled separately

  • is represented by a grid with features populated from data, estimation, or simulation

  • in 2D modeling is commonly called the area of interest

  • the extent and grid cell size are selected based on a trade-off between model accuracy and computational complexity

Used in:

Volume Support#

The spatial volume over which a feature or variable is measured or averaged. Volume support defines the physical extent of a measurement and directly influences variability, smoothing, and scale dependence. In practice, it is often related to (but not identical with) the concept of scale.

Examples include:

Core volume support is:

\[ \pi r_{core}^2 \times l_{core} \]

where \(r_{core}\) is core radius and \(l_{core}\) is core length.

Well Log volume support is:

\[ \pi r_{log}^2 \times l_{log} \]

where \(r_{log}\) is the logging tool radius (or effective radius of investigation) and \(l_{log}\) is the vertical resolution or sampling interval.

Seismic volume support is:

\[ \delta x_{\text{inline}} \times \delta y_{\text{crossline}} \times \delta z_{\text{vertical}} \]

where \(\delta x_{\text{inline}}\) is inline resolution, \(\delta y_{\text{crossline}}\) is crossline resolution, and \(\delta z_{\text{vertical}}\) is vertical resolution.

It is critical to explicitly state volume support when describing data or models:

  • Volume support strongly influences measured statistics and spatial variability.

  • Consistent comparison between datasets requires accounting for differences in support through change-of-support or upscaling methods.

Used in:

Also see:

Volume-Variance Relations#

The relationship between volume support and variance. In general, as the volume support increases, the variance of a feature decreases because larger volumes average over more spatial variability.

Predicting volume-variance relations is central to integrating data collected at different scales and building subsurface models that represent the appropriate level of heterogeneity.

General observations and assumptions:

  • Under linear averaging and stationary conditions, the mean does not change with volume support; only the variance changes.

  • The distribution shape may change with volume support. This should be evaluated empirically. Common approaches include assuming no shape change with an affine correction or applying a distribution-specific correction such as an indirect lognormal correction.

  • Variance reduction is controlled by spatial continuity. Features with shorter correlation ranges experience faster variance reduction as volume support increases, while features with longer ranges retain variability over larger volumes.

Over common changes in subsurface modeling scale, the impact may be significant. Therefore, volume-variance relations should not be ignored.

  • Perfect scale-up accounting is rarely achieved because sufficient data are generally unavailable to fully characterize variability across all scales. This is commonly referred to as the missing scale problem.

  • A model is required to predict how variance changes with volume support.

Common methods to model and apply volume-variance relations include:

  1. Empirical - build a high-resolution model and numerically upscale to the larger volume support. For example,

  • calculate a fine-scale permeability model

  • apply flow simulation to estimate effective permeability over larger block volumes

  1. Power Law Average - a flexible averaging approach for changing support,

\[ z_V = \left[ \frac{1}{n} \sum z_v^{\omega} \right] ^{\frac{1}{\omega}} \]

\(\quad\) where \(\omega\) is the power of averaging:

  • \(\omega = 1\) is arithmetic averaging

  • \(\omega = -1\) is harmonic averaging

  • \(\omega = 0\) is geometric averaging, obtained as the limit as \(\omega \rightarrow 0\)

\(\quad\) The appropriate \(\omega\) may be determined from:

  • theoretical understanding, for example, harmonic averaging of permeability for flow perpendicular to beds

  • numerical upscaling with flow simulation followed by calibration of an effective averaging exponent

  1. Statistical Model - directly adjust statistical properties for the change in volume support. For linear averaging with a stationary variogram model, the Variance Reduction Factor is:

\[ f = 1 - \frac{\overline{\gamma}(v,v)}{\sigma^2} \]

\(\quad\) where \(f\) is the ratio of variance at larger volume support to variance at the original data support:

\[ f = \frac{D^2(v,V)}{D^2(\cdot,V)} = \frac{D^2(v,V)}{\sigma^2} \]

\(\quad\) The variance reduction factor is calculated from,

  • the variogram model representing spatial continuity

  • the original data support

  • the target model volume support

Also see:

Weak Learner#

A predictive machine learning model that performs only slightly better than random prediction.

A weak learner is represented as,

\[ \hat{Y}=\hat{f}_k(X_1,\ldots,X_m) \]

where \(\hat{f}_k\) is the \(k^{th}\) weak learner, \(X_1,\ldots,X_m\) are the predictor features, and \(\hat{Y}\) is the predicted response feature.

Characteristics of weak learners include,

  • simple model structure with limited flexibility

  • high model bias and low model variance

  • prediction performance only marginally better than random

Examples include,

  • decision stump - a decision tree with only one split and two terminal regions

  • simple linear models for complex nonlinear natural systems

Weak learners are the foundation of ensemble methods, especially boosting,

  • boosting sequentially combines many weak learners, where each new learner focuses on correcting the errors of previous learners

  • the combined ensemble can produce a strong predictive model with improved accuracy and generalization

The terms used include,

  • weak predictor - general term for a weak predictive model

  • weak classifier - weak learner applied specifically to classification problems

Used in:

Weight#

A factor that modifies the contribution, influence, or representativity of data or model information. Common uses include,

  1. Data or model scenario weighting to account for representativity or importance, for example, declustering weights used to improve the statistical representativity and reduce bias of resulting statistics.

  2. A trainable model parameter associated with a regression model that controls the contribution of each predictor feature. In linear models, weights correspond to regression coefficients.

  3. A trainable model parameter associated with a neural network connection that controls the influence of information flowing into a node.

Now more on the third use. Information flows forward through the neural network,

  • each node forms a weighted sum of the incoming information, adds a bias, and applies an activation function

\[ a_j=g\left(\sum_{i=1}^{m}w_{ij}x_i+b_j\right) \]

where \(w_{ij}\) is the weight associated with the connection from node \(i\) to node \(j\).

The magnitude and sign of a weight determine how information influences the next node,

  • positive weights reinforce the incoming signal

  • negative weights oppose the incoming signal

  • larger magnitude weights have greater influence on the node output

During training,

While weights are commonly used for neural networks, there are other important uses of the term weights, including,

Used in:

Contrast with:

Also see:

Well Image Log#

A special case of well logs where the well logs are repeated at various azimuthal intervals within the well bore resulting in a 2D (unwrapped) image instead of a 1D line along the well bore. For example, Fullbore formation MicroImager (FMI) with:

  • with 80% bore hole coverage

  • 0.2 inch (0.5 cm) resolution vertical and horizontal

  • 30 inch (79 cm) depth of investigation

can be applied to observe lithology change, bed dips and sedimentary structures.

Used in: TBD

Also see:

Well Log#

Geostatistical Concepts: as a much cheaper method to sample wells that does not interrupt drilling operations, well logs are very common over the wells. Often all wells have various well logs available. For example,

  • gamma ray on pilot vertical wells to assess the locations and quality of shales for targetting (landing) horizontal wells

  • neutron porosity to assess location high porosity reservoir sands

  • gamma ray in drill holes to map thorium mineralization

Well log data are critical to support subsurface resource interpretations. Once anchored by core data they provide the essential coverage and resolution to model the entire reservoir concept / framework for prediction, for example,

  • well log data calibrated by core data collocated with well log data are used to map the critical stratigraphic layers, including reservoir and seal units

  • well logs are applied to depth correct features inverted from seismic data that have location imprecision due to uncertainty in the rock velocity over the volume of interest

Used in: TBD

Also see:

Comments#

This was a basic introduction to geostatistics. If you would like more on these fundamental concepts I recommend the Introduction, Modeling Principles and Modeling Prerequisites chapters from my text book, Geostatistical Reservoir Modeling{cite}`pyrcz2014’.

I hope this is helpful,

Michael

The Author:#

Michael Pyrcz, Professor, The University of Texas at Austin Novel Data Analytics, Geostatistics and Machine Learning Subsurface Solutions

With over 17 years of experience in subsurface consulting, research and development, Michael has returned to academia driven by his passion for teaching and enthusiasm for enhancing engineers’ and geoscientists’ impact in subsurface resource development.

For more about Michael check out these links:

Twitter | GitHub | Website | GoogleScholar | Geostatistics Book | YouTube | Applied Geostats in Python e-book | Applied Machine Learning in Python e-book | LinkedIn

Want to Work Together?#

I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.

  • Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I’d be happy to drop by and work with you!

  • Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!

  • I can be reached at mpyrcz@austin.utexas.edu.

I’m always happy to discuss,

Michael

Michael Pyrcz, Ph.D., P.Eng. Professor, Cockrell School of Engineering and The Jackson School of Geosciences, The University of Texas at Austin

More Resources Available at: Twitter | GitHub | Website | GoogleScholar | Geostatistics Book | YouTube | Applied Geostats in Python e-book | Applied Machine Learning in Python e-book | LinkedIn