Machine Learning Workflow Construction and Coding#
Michael J. Pyrcz, Professor, The University of Texas at Austin
Twitter | GitHub | Website | GoogleScholar | Geostatistics Book | YouTube | Applied Geostats in Python e-book | Applied Machine Learning in Python e-book | LinkedIn
Chapter of e-book “Applied Machine Learning in Python: a Hands-on Guide with Code”.
Cite this e-Book as:
Pyrcz, M.J., 2024, Applied Machine Learning in Python: A Hands-on Guide with Code [e-book]. Zenodo. doi:10.5281/zenodo.15169139 
The workflows in this book and more are available here:
Cite the MachineLearningDemos GitHub Repository as:
Pyrcz, M.J., 2024, MachineLearningDemos: Python Machine Learning Demonstration Workflows Repository (0.0.3) [Software]. Zenodo. DOI: 10.5281/zenodo.13835312. GitHub repository: GeostatsGuy/MachineLearningDemos 
By Michael J. Pyrcz
© Copyright 2024.
This chapter is a summary of Machine Learning Workflow Construction and Coding including essential concepts:
Suggestions for Learning to Code
Basics of Workflow Design
Workflow Building Blocks
YouTube Lecture: check out my lectures on:
Motivation for Machine Learning Workflow Construction and Coding#
You could learn the theory of data science and machine learning. Completely understand probability, statistics, inference and prediction, cross validation for hyperparameter tuning and model checking, but to add value we must code practical, fit-for-purpose workflows that can be widely deployed and used by many others. I’ve done this for many years and my goal is to share some of my experience to help you get started, or hone your skills!
If you don’t already code then please review my reasons for all scientists and engineers to learn some coding. I often share this on social media and in my classes.
Reasons for All Scientists and Engineers to Learn Some Coding#
Here’s Professor Pyrcz’s reasons for all scientists and engineers to learn some coding.
Transparency – no compiler accepts hand waiving! Coding forces your logic to be uncovered for any other scientist or engineer to review.
Reproducibility – run it and get an answer, hand it over to a peer, they run it and they get the same answer. This is a principle of the scientific method.
Quantification – programs need numbers and drive us from qualitative to quantitative. Feed the program and discover new ways to look at the world.
Open-source – leverage a world of brilliance. Check out packages, snippets and be amazed with what great minds have freely shared.
Break Down Barriers – don’t throw it over the fence. Sit at the table with the developers and share more of your subject matter expertise for a better deployed product.
Deployment – share your code with others and multiply your impact. Performance metrics or altruism, your good work benefits many others.
Efficiency – minimize the boring parts of the job. Build a suite of scripts for automation of common tasks and spend more time doing science and engineering!
Always Time to Do it Again! – how many times did you only do it once? It probably takes 2-4 times as long to script and automate a workflow. Usually, worth it.
Be Like Us – it will change you. Users feel limited, programmers truly harness the power of their applications and hardware.
I have decades of Basic, Fortran, Visual Basic for Applications (VBA), C++, R, and Python. My first codes in middle school were written in Basic on a Texas Instruments TI-99/4A home computer and stored on cassette tape. I’ve done a lot of numerical prototypes, workflows, scripts, dashboards and even a few years of full-stack development. Let me put it this way, “canned software” makes me feel trapped, I’m immediately looking for an Application Programming Interface (API) so I can escape and write my own methods, dashboards and visualizations. Yet I remain very practical and humble about coding; therefore, my advice is learn what you need to get the job done. Here some alternatives:
learn to use the workflow automation functionality in canned software. The documentation is usually quite good and this approach not only provides an audit trail for your workflows, but it also is compliant with your organization’s workflow standard and IT department policies.
if the canned software accepts Python, Java, controls then learn some basics in those language to interactive with the objects in your workflows. This will often greatly improve your flexibility to make value adding workflows.
many organizations are starting to prototype and deploy workflows in Python with tools such as Jupyter Notebooks and Jupyter Lab. If this is the case, learn some Python basics, start working with basic Python packages like Numpy, Pandas, Matplotlib, and SciPy. Go online and Google and ask ChatGPT to help with your workflow development. There are so many resources to help, including this e-book and the basics in coding that I share in this chapter.
Don’t be intimidated, I don’t wish (or curse on) you a life of full stack development. Those were days of searching for memory leaks and hanging pointers in C++. This is what I say after all of these experiences,
With Python
I code much less, and get more done!
Modeling Purpose#
The first step in any data science project is to pose the question, what is the purpose of this work? To address this we have to step back and ask:
What is the problem?
What have others done to solve this problem?
What are we going to do to solve this problem?
What can we deploy to solve this problem?
I often tell my students,
If your data science workflow does not ultimately influence a decision, it has not yet delivered its full value.
This is a useful principle for keeping projects focused, but subsurface modeling serves many purposes beyond making an immediate operational decision. It provides a common understanding of the subsurface, quantifies uncertainty, transfers geological knowledge, and ultimately supports better decisions throughout the life of a project.
Although these goals differ, they all rely on the same fundamental workflow:
Subsurface Data
│
▼
Common Earth Model
│
▼
Resource Assessment
│
▼
Uncertainty Models
│
▼
Risk Analysis
│
▼
Development Alternatives
│
▼
Business Decisions
Now let’s review some purposes for modeling. Now these will be focused on my experience in subsurface modeling, but they are transferable to any other type of data science project.
Build a Common Earth Model#
The first goal is to build a shared representation of the subsurface. While this may appear to be “modeling for the sake of modeling,” it is much more than that. The common Earth model provides a platform to,
build a shared understanding of the subsurface across the multidisciplinary project team
integrate all available information into a consistent interpretation
establish what is known, what remains uncertain, and the critical project risks
communicate the subsurface interpretation within and outside the project team
Example model and data visualization from DGI.
Assess Resources#
Estimate the gross volume of interest and the associated spatial distribution of the in-situ subsurface resource.
estimate hydrocarbons or minerals in place
characterize spatial variability in quantity and quality
while not as rigorous as a reserves evaluation, the assessment may include consideration of extraction methods and associated technical constraints
Quantify Resource Uncertainty#
Integrate multiple sources of uncertainty to generate alternative, equally plausible realizations of the subsurface.
characterize uncertainty in resource estimates
identify where additional data would most effectively reduce uncertainty
support development planning and, where appropriate, public resource disclosures
Investigate Geologic Risk#
While uncertainty describes what the subsurface could be, risk considers the consequences of that uncertainty.
Evaluate,
sensitivities — which uncertainties have the greatest impact?
downside risk — what could go wrong and how severe could the consequences become?
upside potential — what favorable outcomes are possible and how can they be increased?
Evaluate Development Strategies#
Compare alternative development scenarios accounting for available information and uncertainty.
Examples include,
primary, secondary, and tertiary recovery processes
drilling sequence and well spacing
completion strategies
facilities design and operating plans
Assess Reserves#
Estimate quantities that geological and engineering data demonstrate with reasonable certainty,
to be recoverable in future years
from a mappable subsurface volume
under existing economic and operating conditions
Resource assessment estimates what exists, while reserve estimation evaluates what can be economically recovered under established reporting standards.
Modeling and calculations are consistent with reporting standards. Image below from SEC.
Transfer Geological Knowledge#
Knowledge developed from mature reservoirs and mining districts can often be transferred to analogous projects.
Examples include,
statistical distributions representing geological uncertainty
spatial trends and nonstationarity
training images for multiple-point geostatistics
geological analogs and image datasets for vision-based deep learning
Transferred geological knowledge is particularly valuable during exploration and early development when direct observations are sparse.
Ultimately: Support Better Decisions#
Every subsurface model should improve one or more decisions by reducing uncertainty, identifying opportunities, or avoiding costly mistakes.
Applications include,
well location selection
mine planning and dig limits
equipment selection
facilities design
economic evaluation
project go / no-go decisions
The objective is not simply to produce an accurate model, but to provide the best available estimate of the subsurface together with its associated uncertainty so decision makers can make informed choices.
For more on these modeling objectives, see Pyrcz and Deutsch (2014). Image below from Mining-Technology.
The Common Machine Learning Workflow#
Once we know the purpose for our data science modeling work, then we can utilize common modeling workflows. For example, for many machine learning projects we follow these steps:
Data Cleaning - there are going to be data issues. Missing data, data with errors, biased data, etc. Often data cleaning is 80% - 90% of the effort on a data science project. This step requires a lot of domain expertise, so I can’t give you too much advice for your case besides,
be patient with this step, rushing data cleaning will usually impact model accuracy and require a lot of repeated work later.
engage with the entire project team, including all experts that know the various data types. Don’t assume that you know all the essential metadata.
Feature Engineering - is about taking the raw features and getting them into the optimum form to build the best possible models. This may include feature transformations, feature projection to new more useful features or even feature selection. My advice is,
don’t work in autopilot and just pass all the available features into your machine learning models and hope the models will sort it out. This typically results in low interpretability, and lower accuracy models.
Inferential Machine Learning - apply inferential machine learning to learn more from your limited sample data about the population. This may include cluster analysis to infer separate populations to model separately.
this is your chance to explore the data to learn new patterns and then update the plan for the next step, predictive modeling
Predictive Machine Learning - train and tune predictive machine learning models based on all the previous work, i.e., with clean data, engineered features, and inferential models about the population(s).
this step gets the most attention, but it is often only about 3% - 5% of the work. If you are spending most of your time on this step that should be a red flag.
Model Checking and Documentation - we always, always check every step in our workflows and after we build our models we check them. You might say, “Why? Didn’t we already train and tune our models?”. You are correct, we did take a lot of care, but we are not done yet. Some ideas:
use explainable AI tools like Shapley values to understand the behaviors of our model.
stress test your model with various interpolation and extrapolation cases to determine when your model breaks.
now, put together great documentation, diagnostics, and training content (you could also cite my e-books and my YouTube lectures). This step is critical for the widest deployment and biggest impact for your work.
Fit-for-Purpose Modeling#
Regardless what data science workflow you build, you need to have a “fit-for-purpose” mentality. What does that mean?
As we discussed, the data science workflow design considers the goals of the workflow
We should also consider the future possible needs for the workflow, i.e., is it extendable?, could it be rapidly updated?, and will you be able to keep up if it is a hit in your organization?
Also we must always account for the resources, time and people (expertise) available in our organization. For example, deploy a simple workflow if your organization does not have a lot of data science expertise and compute resources available
We can’t have it all! This image has been very helpful to me over my career, it is the good, fast, cheap diagram, that illustrates the project trade off that you can only pick two and cannot have all three!
Modeling Constraints#
Let’s talk a bit more about modeling constraints. Every organization has limitations, this is critical to capital stewardship and profitability. Here’s some of the common constraints,
Professional Time - work hours available limited by workforce, number of projects and project timelines schedules. Our time is valuable and within the organization there is an accounting of the cost to maintain each project team member, full time equivalent (FTE), for a period of time. From this we can think about what activities are value adding to offset this cost.
Organization Capability - some organizations call this operational capability, it is the skill sets of the professionals that are available for the project. Yes, you may have a world renowned deep learning expert in your project team, but their time is likely split over multiple projects along with mentorship and leadership responsibilities.
Computational Facilities - this includes the computational storage and processing of the hardware, along with the software available for the project. You may have access to a specific numerical simulation software, but with a limited number of seats shared over the organization, and it would be rare to be able to run our deep learning convolution models over massive 3D model spaces.
Data Collection - you will not get all the data that you want for your data science project. Number of samples will usually be limited, and you may not have all the features that you want. Sometimes there may even be controls on how data is moved, stored and used that may limit your work. May I offer some sagely advice? Be very careful about this! Always have permission for any use and release of data, including work derived from data.
Total Budget - if you put this all together you get the ultimate limitation, how much can be spent on the project. This is the ultimate limit on all the previous, professional time, computational resources and data collection
Modeling Strategies#
There are various overarching modeling strategies. I will only mention a couple here.
Data Integration#
Subsurface modeling is fundamentally a data integration problem. Modelers sit at the intersection of multidisciplinary teams and combine observations, physical understanding, and expert knowledge into a consistent numerical representation of the subsurface.
The challenge includes,
integrating drill holes, well logs, seismic surveys, production data, geological concepts, and engineering physics
reconciling observations collected over vastly different spatial and temporal scales
accounting for varying levels of measurement uncertainty (“hard” versus “soft” data)
filling gaps where observations are sparse or entirely absent
reconciling contradictory evidence from multiple data sources
The result is a consistent numerical representation of subsurface properties that honors all available information.
Top-down Modeling#
I like this approach, it is very conservative with elements of fail fast and fit-for-purpose modeling. It proceeds with these steps.
Start with the simplest model possible and forecast with the model
Incrementally add detail and complexity to the model, forecast and access the change in the forecast due to added complexity
Iterate until additional detail and complexity does not impact the forecast
For geomodelling this is applied as starting with a simple grid framework and some basic large scale heterogeneity, e.g., major geological discontinuities and barriers. Apply numerical simulation and evaluate the model performance. Then add in more medium scale heterogeneity within the large scale framework and repeat the numerical simulation and evaluation with a focus on the change. Repeat this process until the additional complexity does not impact the numerical simulation results.
How does top-down modeling apply to machine learning? In a way we already do it, when we tune the hyperparameters by testing a simple machine learning model and then incrementing the complexity until the test error is minimized. We can go further and integrate top-down modeling into our general approach by asking, what is the simplest model that gets us an accurate enough model, because this model will likely be more interpretable and practical to deploy.
Modeling for Discomfort#
Mark Bentley proposed the concept of “modeling for comfort” [Ben16] with the suggestion that models may have become tools for verification of a decision already partially or fully made. To defend against this tendency, Bentley recommends that we model for discomfort!
Stress test our current concepts and sensitivity on decision-making
Bentley states that by doing this we can identify remaining up-side potential and while securing against worst case. Also this is a great approach to recognize and mitigate our biases.
I call this the Mythbusters approach after the science and engineering TV show with Jaime and Adam. My kids grew up watching Mythbusters and I loved the show also. They took something from a movie or history and tested it. For example,
will focusing many polished shields start a wooden ship on fire
will putting a car in reverse while driving cause the transmission to fall out
can a high pitch sound break a glass
How do we apply modeling for discomfort or the Mythbusters approach in our model construction. Simple, stress test your models. Consider the following,
your model predicts very well, but can you find cases for which your model is inaccurate
your model is robust to outliers, add more outliers and observe the limit in this robustness
your model preserves information while reducing the dimensionality of the problem, add more dimensions
your model runs fast enough in 2D, try it in 3D
This is the equivalent to Jaime and Adam failing to see a failure and deciding to add more pressure, more velocity, more… until it fails.
Learning to Develop Workflows in Python#
I have taught data science to hundreds of engineers and geoscientists, many of them had no programming experience and are now doing data science in Python. This is awesome and I’m excited to help. To support these students I have developed a verity of resources that are helpful,
Data Science Basics in Python code walk-throughs with GeostatsGuy
I selected some of the building blocks of machine learning workflows and built out live code presentations (with the Rise Python package) and recorded my walk-through and included it as a playlist[Pyr24c] on my YouTube channel. Anyone can open up the workflows (links in the comments and follow along).
Machine Learning Course on YouTube
My entire machine learning course is available as a playlist[Pyr24c] on my YouTube channel[Pyr24d]. Note only does it include all the recorded lectures, but also links in the comments to all the codes, including well-documented workflows and interactive dashboards in Python. I provide many realistic examples of working with data, building machine learning models, visualizing and checking the data and models, etc.
Applied Machine Learning in Python, a Hands-on Guide with Code
Of course, this book is full of resources, including a lot of example code. In fact, in many cases I included additional comments and descriptions to help those new to coding.
You’ve Got This
I’ve seen many quickly learn the basic coding skills to build machine learning workflows.
About the Author#
Michael Pyrcz is a professor in the Cockrell School of Engineering, and the Jackson School of Geosciences, at The University of Texas at Austin, where he researches and teaches subsurface, spatial data analytics, geostatistics, and machine learning. Michael is also,
the principal investigator of the Energy Analytics freshmen research initiative and a core faculty in the Machine Learn Laboratory in the College of Natural Sciences, The University of Texas at Austin
an associate editor for Computers and Geosciences, and a board member for Mathematical Geosciences, the International Association for Mathematical Geosciences.
Michael has written over 100 peer-reviewed publications, a Python package for spatial data analytics, co-authored a textbook on spatial data analytics, Geostatistical Reservoir Modeling and author of two recently released e-books, Applied Geostatistics in Python: a Hands-on Guide with GeostatsPy and Applied Machine Learning in Python: a Hands-on Guide with Code.
All of Michael’s university lectures are available on his YouTube Channel with links to 100s of Python interactive dashboards and well-documented workflows in over 40 repositories on his GitHub account, to support any interested students and working professionals with evergreen content. To find out more about Michael’s work and shared educational resources visit his Website.
Want to Work Together?#
I hope this content is helpful to those that want to learn more about subsurface modeling, data analytics and machine learning. Students and working professionals are welcome to participate.
Want to invite me to visit your company for training, mentoring, project review, workflow design and / or consulting? I’d be happy to drop by and work with you!
Interested in partnering, supporting my graduate student research or my Subsurface Data Analytics and Machine Learning consortium (co-PIs including Profs. Foster, Torres-Verdin and van Oort)? My research combines data analytics, stochastic modeling and machine learning theory with practice to develop novel methods and workflows to add value. We are solving challenging subsurface problems!
I can be reached at mpyrcz@austin.utexas.edu.
I’m always happy to discuss,
Michael
Michael Pyrcz, Ph.D., P.Eng. Professor, Cockrell School of Engineering and The Jackson School of Geosciences, The University of Texas at Austin
More Resources Available at: Twitter | GitHub | Website | GoogleScholar | Geostatistics Book | YouTube | Applied Geostats in Python e-book | Applied Machine Learning in Python e-book | LinkedIn
Comments#
This was a basic description of machine learning workflow construction and coding. Much more could be done and discussed, I have many more resources. Check out my shared resource inventory and the YouTube lecture links at the start of this chapter with resource links in the videos’ descriptions.
I hope this was helpful,
Michael