Engineering Modeling

Tolerance Analysis using Monte Carlo (Part 11 / 13)

Karl - CB Expert

Share:

Print

Rate article:

5.0
Rate this article:
5.0

How do Monte Carlo analysis results differ from those derived via WCA or RSS methodologies? Let us return to the one-way clutch example and provide a practical comparison in terms of a non-linear response. From the previous posts, we recall that there are two system outputs of interest: stop angle and spring gap. These outputs are described mathematically with response equations, as transfer functions of the inputs.

What options are there to run Monte Carlo (MC) analysis on our laptops? There are quite a few (a topic that invites a separate post). For the everyday user, it makes good sense to utilize one that works with existing spreadsheet logic, as in an Excel file. There are three movers and shakers in the Monte Carlo spreadsheet world which one might consider: Crystal Ball, ModelRisk and @Risk. (If you examine our offerings, www.crystalballservices.com, you will find we sell the first two.) For the purposes of this series, we will use Crystal Ball (CB) but will return to ModelRisk in the near future.

As with the RSS approach, it makes sense to enter the transfer function logic into an Excel spreadsheet where equations define the system output as linked to the inputs located in other cells. (Please refer to the "One-Way Clutch with Monte Carlo" spreadsheet model in our model archives. Figure 11-1 contains a snapshot of this spreadsheet.) The inputs and outputs are arranged in vertical order. Each of the inputs has a nominal and tolerance value associated with them, along with a desired Sigma level. As stated in previous posts, it will be assumed the suppliers of these inputs are 3-sigma capable and that their behaviors are normally distributed. Going forward, this means the standard deviations are assumed to be one-third of the tolerance value.

The noticeable difference between this spreadsheet and the previous one (focused only on WCA and RSS results) is the coloring of the cells in the column titles "Values." The input "values" are colored green while the output "values" are colored blue. In the Crystal Ball (CB) world, these colors indicate a probabilistic behavior has been assigned to these variables. If we click on one of the green cells and then the "Define Assumption" button in the CD ribbon bar, a dialog box opens to reveal which distribution was defined for that input. The user will find that they are all normal distributions and the parameters (mean and stdev) are cell-referenced to the adjacent "Nominal," "Tolerance" and "Sigma" cells within the same row. (Note the standard deviation is calculated as the tolerance divided by the sigma. If this is not visible in your CB windows, hit CTL-~.)

We can run the simulation as is. But before we do, an important MC modeling distinction should be made. In the RSS derivations, we assumed that the contributive nature of both bearings (bought in batches from the same supplier) were equal and therefore assumed the same variable behavior for both diameters as one singular variable (dB1) in the output transfer functions. However, this assumption in the transfer function should not be used for the transfer functions in the Monte Carlo analysis.

Re-examine the case of the classic stack-up tolerance example from my first few posts (see Figure 11-2). Note that for the transfer function, I have distinguished each block width as a separate input variable. Had I assumed the same input variable for all three (so that the output gap transfer function is cavity width minus three times a singular block width), I would have invited disaster. Explicit in the incorrect transfer function is an assumption that all three block widths will vary in synchronicity. If the block width is randomly chosen as greater than the nominal by the MC simulation, all three block widths would be artificially larger than their nominals. Obviously, this is not what would happen in reality. The simulation results would overpredict variation. Therefore, the MC modeler must assign different variables for all block widths, even though the probability distributions that define their variation are one and the same.

To run the simulation, the user simply clicks on the "Start" button in the CB ribbon bar. Immediately obvious should be the CB Forecast windows that pop up in the forefront to display the simulation as it runs. The visual result is output histograms that show the variation with respect to the spec limits (LSL & USL), as seen in Figure 11-3. CB also can place particular output results values (such as mean, stdev, capability metrics and percentiles) into defined cells (Cells I12:AK13) via Forecast options.

Here is the moment of truth. How did MC analysis compare to the WCA and RSS results? For an even better "apples-to-apples" comparison, the analyst may decide to extract percentiles associated with the 99.73% range (3 sigmas to either side), rather than calculate them from mean and standard deviation. If there is any non-normal behavior associated with the output, MC analysis can extract the values corresponding to the 3-sigma range, providing more accuracy on the extreme range despite non-normality. The same cannot be said of either WCA or RSS. ("Min Values" and "Max Values" for all three methods are displayed in Columns J & K.)

Figure 11-4 summarizes the range comparisons on both system outputs for the three methods. Surprisingly enough, MC provides a wider range of stop angle extreme values than RSS but less wide than the conservative WCA approach. (RSS and MC agree pretty much on spring gap range while still being less than the WCA range.) The reason for the difference in stop angle extreme ranges is related to a difference in predicted standard deviations. The MC method predicts an output standard deviation that is two orders of magnitude greater than the RSS output standard deviation. (Your results may vary based on number of trials and random seed values but should be approximately the same.) The approximations based in RSS scribes can sometimes be too relaxed, especially if linearity and normality assumptions are violated. They can be too liberal and would paint a rosy picture for dimensional quality predictions.

Is that the only reason to prefer MC analysis over RSS? Follow my next post as we revisit the topic of sensitivities and sensitivity contributions as they apply in the MC world.

Creveling, Clyde M., Tolerance Design: A Handbook for Developing Optimal Specifications (1997); Addison Wesley Longman, pp. 163-167.

Comments

Collapse Expand Comments (0)
You don't have permission to post comments.

Oracle Crystal Ball Spreadsheet Functions For Use in Microsoft Excel Models

Oracle Crystal Ball has a complete set of functions that allows a modeler to extract information from both inputs (assumptions) and outputs (forecast). Used the right way, these special Crystal Ball functions can enable a whole new level of analytics that can feed other models (or subcomponents of the major model).

Understanding these is a must for anybody who is looking to use the developer kit.

Why are analytics so important for the virtual organization? Read these quotes.

Jun 26 2013
6
0

Since the mid-1990s academics and business leaders have been striving to focus their businesses on what is profitable and either partnering or outsourcing the rest. I have assembled a long list of quotes that define what a virtual organization is and why it's different than conventional organizations. The point of looking at these quotes is to demonstrate that none of these models or definitions can adequately be achieved without some heavy analytics and integration of both IT (the wire, the boxes and now the cloud's virtual machines) and IS - Information Systems (Applications) with other stakeholder systems and processes. Up till recently it could be argued that these things can and could be done because we had the technology. But the reality is, unless you were an Amazon, e-Bay or Dell, most firms did not necessarily have the money or the know-how to invest in these types of inovations.

With the proliferation of cloud services, we are finding new and cheaper ways to do things that put these strategies in the reach of more managers and smaller organizations. Everything is game... even the phone system can be handled by the cloud. Ok, I digress, Check out the following quotes and imagine being able to pull these off without analytics.

The next posts will treat some of the tools and technologies that are available to make these business strategies viable.

Multi-Dimensional Portfolio Optimization with @RISK

Jun 28 2012
16
0

Many speak of organizational alignment, but how many tell you how to do it? Others present only the financial aspects of portfolio optimization but abstract from how this enables the organization to meets its business objectives.  We are going to present a practical method that enables organizations to quickly build and optimize a portfolio of initiatives based on multiple quantitative and qualitative dimensions: Revenue Potential, Value of Information, Financial & Operational Viability and Strategic Fit. 
                  
This webinar is going to present these approaches and how they can be combined to improve both tactical and strategic decision making. We will also cover how this approach can dramatically improve organizational focus and overall business performance.

We will discuss these topics as well as present practical models and applications using @RISK.

Reducing Project Costs and Risks with Oracle Primavera Risk Analysis

.It is a well-known fact that many projects fail to meet some or all of their objectives because some risks were either: underestimated, not quantified or unaccounted for. It is the objective of every project manager and risk analysis to ensure that the project that is delivered is the one that was expected. With the right know-how and the right tools, this can easily be achieved on projects of almost any size. We are going to present a quick primer on project risk analysis and how it can positively impact the bottom line. We are also going to show you how Primavera Risk Analysis can quickly identify risks and performance drivers that if managed correctly will enable organizations to meet or exceed project delivery expectations.

.

 

Modeling Time-Series Forecasts with @RISK


Making decisions for the future is becoming harder and harder because of the ever increasing sources and rate of uncertainty that can impact the final outcome of a project or investment. Several tools have proven instrumental in assisting managers and decision makers tackle this: Time Series Forecasting, Judgmental Forecasting and Simulation.  

This webinar is going to present these approaches and how they can be combined to improve both tactical and strategic decision making. We will also cover the role of analytics in the organization and how it has evolved over time to give participants strategies to mobilize analytics talent within the firm.  

We will discuss these topics as well as present practical models and applications using @RISK.

The Need for Speed: A performance comparison of Crystal Ball, ModelRisk, @RISK and Risk Solver


Need for SpeedA detailed comparison of the top Monte-Carlo Simulation Tools for Microsoft Excel

There are very few performance comparisons available when considering the acquisition of an Excel-based Monte Carlo solution. It is with this in mind and a bit of intellectual curiosity that we decided to evaluate Oracle Crystal Ball, Palisade @Risk, Vose ModelRisk and Frontline Risk Solver in terms of speed, accuracy and precision. We ran over 20 individual tests and 64 million trials to prepare comprehensive comparison of the top Monte-Carlo Tools.

 

Excel Simulation Show-Down Part 3: Correlating Distributions

Escel Simulation Showdown Part 3: Correlating DistributionsModeling in Excel or with any other tool for that matter is defined as the visual and/or mathematical representation of a set of relationships. Correlation is about defining the strength of a relationship. Between a model and correlation analysis, we are able to come much closer in replicating the true behavior and potential outcomes of the problem / question we are analyzing. Correlation is the bread and butter of any serious analyst seeking to analyze risk or gain insight into the future.

Given that correlation has such a big impact on the answers and analysis we are conducting, it therefore makes a lot of sense to cover how to apply correlation in the various simulation tools. Correlation is also a key tenement of time series forecasting…but that is another story.

In this article, we are going to build a simple correlated returns model using our usual suspects (Oracle Crystal Ball, Palisade @RISK , Vose ModelRisk and RiskSolver). The objective of the correlated returns model is to take into account the relationship (correlation) of how the selected asset classes move together. Does asset B go up or down when asset A goes up – and by how much? At the end of the day, correlating variables ensures your model will behave correctly and within the realm of the possible.

Copulas Vs. Correlation

Copulas and Rank Order Correlation are two ways to model and/or explain the dependence between 2 or more variables. Historically used in biology and epidemiology, copulas have gained acceptance and prominence in the financial services sector.

In this article we are going to untangle what correlation and copulas are and how they relate to each other. In order to prepare a summary overview, I had to read painfully dry material… but the results is a practical guide to understanding copulas and when you should consider them. I lay no claim to being a stats expert or mathematician… just a risk analysis professional. So my approach to this will be pragmatic. Tools used for the article and demo models are Oracle Crystal Ball 11.1.2.1. and ModelRisk Industrial 4.0

Excel Simulation Show-Down Part 2: Distribution Fitting

 

One of the cool things about professional Monte-Carlo Simulation tools is that they offer the ability to fit data. Fitting enables a modeler to condensate large data sets into representative distributions by estimating the parameters and shape of the data as well as suggest which distributions (using these estimated parameters) replicates the data set best.

Fitting data is a delicate and very math intensive process, especially when you get into larger data sets. As usual, the presence of automation has made us drop our guard on the seriousness of the process and the implications of a poorly executed fitting process/decision. The other consequence of automating distribution fitting is that the importance of sound judgment when validating and selecting fit recommendations (using the Goodness-of-fit statistics) is forsaken for blind trust in the results of a fitting tool.

Now that I have given you the caveat emptor regarding fitting, we are going to see how each tools offers the support for modelers to make the right decisions. For this reason, we have created a series of videos showing comparing how each tool is used to fit historical data to a model / spreadsheet. Our focus will be on :

The goal of this comparison is to see how each tool handles this critical modeling feature.  We have not concerned ourselves with the relative precision of fitting engines because that would lead us down a rabbit hole very quickly – particularly when you want to be empirically fair.

RESEARCH ARTICLES | RISK + CRYSTAL BALL + ANALYTICS