Bayesian Reasoning using R (Part 2) : Discrete Inference with Sequential Data
Analytics Articles

Bayesian Reasoning using R (Part 2) : Discrete Inference with Sequential Data

Robert Brown

How I Learned to Think of Business as a Scientific Experiment

Share:

Print

Rate article:

2.5
Rate this article:
2.5
Bayesian Reasoning: Discrete Inference with Sequential Data

In my last article on this topic, I showed that considering background information can play a significant role in helping us make less biased judgments. What I hope to show now is that while we learn by updating the information we have through experience, limited experiences can often lead to prejudices about the way we interpret the world; but even broad and deep experience should rarely lead us to certain conclusions.

To get started, imagine playing a game in which someone asks you to infer the number of sides of a die based on the face numbers that show up in repeated throws of the die. The only information you are given beforehand is that the actual die will be selected from a set of seven die having these number of faces: (4, 6, 8, 10, 12, 15, 18). Assuming you can trust the person who reports the outcome on each throw, after how many rolls of the die will you be willing to specify which die was chosen?

Let’s use the R programming language to help us think through the problem. Start by specifying the set of the die possibilities such that each number represents the number of sides of a given die. (You might also want to refer to my previous article on Bayesian analysis to familiarize yourself with some of the terminology that follows.)

[Note: If you’re not into programming, you can skim past the crunchy geeky bits and still get the flavorful creamy nougat inside. But you have to practice self control to get the delayed satisfaction beneath that layer. You can do it!]

die <- c(4, 6, 8, 10, 12, 15, 18)
n.die <- length(die)

The game master selects a die to be in play but keeps the choice hidden from us. As I’ll explain in more detail later, the game master is the aggregate market or nature, and by “in play,” I mean that some condition that we currently care about in that environment is the case.

die.choice <- 12

Given that we are tasked to infer which die is chosen, and we don’t know any other information other than the number of die in the set (before we observe the outcome of any die toss), any initial guess is as good as the other. Furthermore, we can describe our inclination to believe one situation is the case to the exclusion of any others with a number between 0 and 1, where 0 means that we don’t believe at all, and 1 means that we certainly believe with no reservations. We’ll call this initial guess our prior probability (or our initial degree of belief) that any one of the dice is in play. What number should we assign for each of the possibilities? Well, since we know that any guess is as good as any other, that the possibilities are mutually exclusive, but that any one of the possibiities we’ve identified will certainly be the case, a little thought should tell us that 1/(number of dice) should be the prior probability that we assign to all the possibilities…at least initially. (Some really smart mathematicians described this as the principle of insufficient reason) As we proceed, we’ll discuss how to update these probabilities as we get more information as we aggregate the outcomes of the toss of the die. In R, we apply this probability across a vector with a size equal to the set of die possibilities.

initial.prob <- 1/n.die
initial.probs <- rep(initial.prob, n.die)
[1] 0.1428571 0.1428571 0.1428571 0.1428571 0.1428571 0.1428571 0.1428571

Before we start the process of observing the die outcomes as they are tossed, create an index to keep track of those trials as they occur. Let’s start with 20 trials.

n.trials <- 20
trials <- 0:n.trials

Initialize the trials outcome vector. The first trial outcome (i.e., trial=0) will be NA because we don’t conduct a trial until trial=1.

trial.outcomes <- rep(0, n.trials+1)
trial.outcomes[1] <- NA

Establish the probability that any face of a given die occurs, assuming that each die is fair. For any die that could be in play, the probability that any of its sides occurs is 1/(number of sides). This is another application of the principle of insufficient reason.

outcome.prob <- 1/die

We will keep track of the three key probabilities in three separate arrays as the trials reveal their outcomes. We initialize each array here. The first two arrays are the same size with shape Array(m x n) = (number of trials) x (number of possibilities). The third is a vector of length (number of trials).

  1. This array corresponds to the probability typically denoted as prob(evidence | hypothesis) or our “likelihood” function. For our exercise, we interpret this as the probability that we observe a revealed outcome given that the choice of a certain die is in play. The first values at trial=0 are our prior probabilities.
prob.outcome.gv.choice <-
  array(rep(0, (n.trials+1) * n.die),
        dim = c((n.trials+1), n.die))
prob.outcome.gv.choice[1, ] <- initial.probs
colnames(prob.outcome.gv.choice) <- die
  1. This array corresponds to the probability typically denoted as prob(hypothesis | evidence) or our posterior probability. For our exercise, we interpret this as the degree of belief that a certain die is in play given the cumulative observations of outcomes.
prob.choice.gv.outcome <- prob.outcome.gv.choice
  1. This vector corresponds to the sum of probabilities of outcomes on each trial. In each trial, it’s the normalizing constant. The first value at trial=0 is 1 since it is the sum of the initial prior probabilities or (number of possibilities) * (1/(number of possibilities)).
marginal.prob <- sapply(trials, function(t) sum(prob.outcome.gv.choice[t+1, ]))

Before we start our simulated experiment, let’s think a bit more about what each of those three arrays mean. The first array tells us how likely it is that we should observe a given outcome if a die of a certain size is in play. If the die is four-sided, the likelihood that we would observe a 1, 2, 3, or 4 is 1/4; likewise, for a 12-sided die, any one of the faces would have a likelihood of 1/12. The second array keeps track of our degrees of belief that one of the die types was selected for play. The first set of values in this array will be our prior probabilities, which are all equal, but they will change as we observe the outcomes of the tosses. Hopefully, they will do so such that our confidence that one die type is the case moreso than the other possibilities. Finally, the third vector is the sum of the likelihoods for any outcome that occurs. For example, if a 9 appears on a toss, the likelihood that we would observe a 9 for any of the given generating possibilities is…

df.likelihood <- data.frame(die, ifelse(9>=die, 0, outcome.prob))
colnames(df.likelihood) <- c("Die Sides", "Likelihood Given 9")
df.likelihood %>%
  kable() %>%
  kable_styling(bootstrap_options = "striped",
                full_width = F) %>%
  column_spec(1, bold = T, border_right = T, width = "2.25cm") %>%
  column_spec(2, width = "3.75cm")
Die Sides Likelihood Given 9
4 0.0000000
6 0.0000000
8 0.0000000
10 0.1000000
12 0.0833333
15 0.0666667
18 0.0555556

The sum of those values in the right column would be r sum(df.likelihood$Likelihood) for the trial on which we might observe that 9.

Run the simulation by stepping across the trials index, starting at the second position (i.e., trials=1) by skipping the initial position (i.e., trials=0). The term trials[-1] is a reduced form of the trials index that removes the initial position. As we step through each trial, we will update our posterior probability as

posterior = previous_posterior * observed_outcome_likelihoods / sum(observed_outcome_likelihood)

for (t in trials[-1]) {
  # Generate a new outcome for each trial based on the die in play.
  trial.outcomes[t+1] <- round(runif(1, 1, die.choice))
  # Calculate the new prior probability for each possibility on each new trial.
    prob.outcome.gv.choice[t+1, ] <-
      (1 - (trial.outcomes[t+1] > die)) *
      prob.choice.gv.outcome[t, ] * outcome.prob
    # Calculate the marginal probability in each trial.
    marginal.prob[t+1] <- sum(prob.outcome.gv.choice[t+1, ])
    # Calculate the updated posterior for each possibility on each trial.
    prob.choice.gv.outcome[t+1, ] <- prob.outcome.gv.choice[t+1, ] /
      marginal.prob[t+1]
}
df.trial.outcomes <- data.frame(trials, trial.outcomes)
colnames(df.trial.outcomes) <- c("trial", "outcome")
df.trial.outcomes %>%
  kable(caption = "Table of trials and the associated observed outcome on each.") %>%
  kable_styling(bootstrap_options = "striped",
                full_width = F) %>%
  column_spec(1, bold = T, border_right = T) %>%
  column_spec(1:2, width = "3cm")
Table of trials and the associated observed outcome on each.
trial outcome
0 NA
1 6
2 5
3 5
4 6
5 4
6 4
7 1
8 5
9 9
10 3
11 10
12 10
13 4
14 6
15 5
16 11
17 4
18 4
19 9
20 3

Now, redefine the prob.choice.gv.outcome array as a data frame for plotting with ggplot2.

df.prob.choice.gv.outcome <- as.data.frame(prob.choice.gv.outcome,
                                           colnames = die)

Transform the data frame df.prob.choice.gv.outcome into a data frame with relational structure such that trial is the left outermost column, die possibility is the next column, and the values are the final right column.

melt.prob.choice.gv.outcome <- melt(cbind(trials, df.prob.choice.gv.outcome),
                                    id.vars="trials")

Plot the probabilities of the die possibilities as they evolve across the trials.

plot.prob.trials <-
  ggplot(data = melt.prob.choice.gv.outcome,
                    aes(x = trials,
                        y = value,
                        group = variable,
                        colour = variable))
plot.prob.trials <- plot.prob.trials + geom_line()
plot.prob.trials <- plot.prob.trials + geom_point()
plot.prob.trials <- plot.prob.trials + theme(axis.text = element_text(size=14),
                                             axis.title = element_text(size=14,face="bold"),
                                             title = element_text(size=16,face="bold"))
plot.prob.trials <- plot.prob.trials + labs(title = "Probability That a Given Die is in Play",
                                            x = "Trials",
                                            y = "Probability"
                                            )
plot(plot.prob.trials)

We can also present this as slices from some selected steps after the first trial across the probabilities of the die possibilities. Determine the rational ordering of each possibility in each slice.

steps <- 5
check.steps <- 1:steps
trial.slices <- c(0, 1, ceiling(check.steps*n.trials/steps))
sub.prob.choice.gv.outcome <- n.die * length(trial.slices)
melt.prob.choice.gv.outcome2 <-
  melt.prob.choice.gv.outcome[1:sub.prob.choice.gv.outcome, ]
for (t in 1:length(trial.slices)) {
melt.prob.choice.gv.outcome2[(t-1) * n.die + (1:n.die), ] <-
  melt.prob.choice.gv.outcome[melt.prob.choice.gv.outcome$trials==trial.slices[t], ]
}
plot.selection <-
  ggplot(data = melt.prob.choice.gv.outcome2, aes(x = variable,
                                                  y = value,
                                                  fill = variable))
plot.selection <- plot.selection + geom_bar(stat="identity")
plot.selection <- plot.selection + facet_grid(. ~ trials)
plot.selection <- plot.selection + theme(axis.text.x = element_text(size=12),
                                         axis.text.y = element_text(size=14),
                                         axis.title = element_text(size=14,face="bold"),
                                         title = element_text(size=16,face="bold"))
plot.selection <- plot.selection + labs(title = "Die Probability at Selected Trials",
                                        x = "Die Possibilities",
                                        y = "Probability")
plot(plot.selection)

Observations and Conclusions (for as long as they matter)

So, what does all this mean, beside learning some nifty R code? How does it relate to business and our intuition?

Let’s start with our prior probability. In cases in which we can account for the possibilities of outcomes and for which we have no other information than just their designations, we should see that any outcome should be thought of as equally likely. What if we arrange those outcomes from least desirable to most desirable. Instead of overworking our anxiety muscle that the worst case will occur, we can alleviate some of our worry by realizing that the likelihood of better outcomes will most likely be the case. There is an obverse side to this observation, though. Instead of building up our hopes and dreams that the best case will occur, we might temper that enthusiasm a bit by realizing that a less than best case will most likely be the case. From the outset, we should be less inclined to allocate a bulk of our resources on those extreme ends…until we get more information. Regardless, at the beginning, we should be reserved about letting our emotions dominate our minds on the basis of the possibility of extreme outcomes. We should acknowledge they exist, but we shouldn’t purchase worry or exuberance until either are warranted. The world of our future probably lies close to the median.

Now, as we begin to step through the trials (and tribulations of life), we see that our degrees of belief about the type of die that could be in play evolves. In fact, the very first trial obviates the need for hypothesizing that a four sided die could be the case because we observed a 6. Under no circumstances can a four-sided die produce a 6; thus, our degree of belief in 4 collapses to 0. This is an example of disconfirming evidence, a hallmark of the scientific process, the purpose of which is to find reasons not to believe that a postulated belief is true. In this case, we found one. There is no more need to waste time planning on 4-sided die being the cause of the events we observe.

As our evidence accumulates, two possibilities (the 6- or 8-sided die) help us to increase our inclination that either is the case as the other possibilities melt away. This is how experience helps us to learn. As we see more and more examples of evidence for the possibility of a given causal explanation, we increasingly accept the dominate case to be true. But as you see, this only works for a short while. There is a dark side to experience, and it’s called prejudice. In our experiment here, by the end of 8 trials, we would have reached a strong degree of belief that the 6-sided die is the causal mechanism for the events we observe. But then a 9 shows up, and our now possibly cherished belief (the 6-sided die explains my world) is dashed against the rocks. Think about this for a moment. There are many events that don’t occur with a large frequency, and we may be fortunate or unfortunate to observe as many as 8 or 10 of them in our lifetimes. But how many times have we either exhorted or warned others on our “vast experience” that we know the underlying causes for what they may be facing? Let’s admit it here: age confers on us a certain amount of experience, some of which may be simply prejudice.

As our experiment proceeds, we experience again that another explanation for the world increases its dominance, only to be shot down like our last strongly held belief. Will there ever be any relief to this cycle of learning and relearning? Maybe. Maybe not. Notice what happens with the 15-sided die. It’s associated degree of belief declines almost to 0, resurrects again on trial #9, declines almost to 0, and then pulls a third appearance on trial #16, but it never quite makes it back down to 0 even as the associated degree of belief in the 12-sided die approaches 0.99. This is the exact same pattern we observed in the 10- and 12-sided die, leading us to believe that either most likely would not explain our world until that pesky disconfirming evidence showed up to disabuse us of our belief. So, after 20 trials, should we be comfortable believing that the 12-sided die is, in fact, the game piece chosen by the game master? A 0.99 degree of belief, as most of all the other explanations have been disconfirmed, empirically and rationally discarded, is a good reason to bet on that idea.

But don’t get cocky, kid. Although we know in the omniscient setup that the 12-sided die is the one in play, we don’t know that in our simulated world. The 15- and 18-sided die have not yet shown us evidence of their existence. In our simulated world, we might think this could be due to the relatively low probability of any number greater than 12 appearing. And so, another explanation always lurks, awaiting to disconfirm our currently held cherished belief in 12-sided die.

Experience is certainly a good thing. It’s how we learn, but it doesn’t account for everything. Simple simulations like the preceding one can help us understand that and help us think about how we can address that constraint. But it won’t be easy. Life never is. Consider the following two conumdrums.

  1. We probably won’t be able to account for all the possible explanations for the effects we observe. One of the best ways to address this is to draw on a diversity of experiences and perspectives of others to consider what we alone never might.
  2. It’s possible that what we see isn’t what we should believe. Eye witness accounts are notoriously unreliable, especially in high stress, high tempo environments. Our measurement devices are sometimes miscalibrated or inherently inaccurate. Suppose, for example, that each time we read the number on the tossed die, there is a intrinsic tendency to misread that outcome, maybe due to fatigue, ailing eyes, or the like. Or suppose we are using a machine that reads a 9 as a 4 every 10th time. Sometimes the possibilities can be mislabeled, either by accident, laziness, or intent. In the case that lack of accuracy is a significant concern, we possibly will never be able to disconfirm any hypothesis altogether.
LS0tCnRpdGxlOiAnQmF5ZXNpYW4gUmVhc29uaW5nOiBEaXNjcmV0ZSBJbmZlcmVuY2Ugd2l0aCBTZXF1ZW50aWFsIERhdGEnCmF1dGhvcjogIlJvYmVydCBELiBCcm93biBJSUkiCmRhdGU6ICI4LzMvMjAxOCIKb3V0cHV0OgogIGh0bWxfZG9jdW1lbnQ6IGRlZmF1bHQKICBodG1sX25vdGVib29rOiBkZWZhdWx0Ci0tLQoKYGBge3IgbGlicmFyaWVzLCBlY2hvID0gRkFMU0V9CmxpYnJhcnkoZ2dwbG90MikKbGlicmFyeShyZXNoYXBlMikKbGlicmFyeShrbml0cikKbGlicmFyeShrYWJsZUV4dHJhKQpzZXQuc2VlZCg5OCkKYGBgCgpgYGB7ciBzZXR1cCwgaW5jbHVkZT1GQUxTRX0Ka25pdHI6Om9wdHNfY2h1bmskc2V0KGVjaG8gPSBUUlVFKQpgYGAKCkJhY2sgdG8gW0luY2l0ZSEgRGVjaXNpb24gVGVjaG5vbG9naWVzXShodHRwOi8vd3d3LmluY2l0ZWRlY2lzaW9udGVjaC5jb20pLgoKIyMgT3IsIE9uZSBXYXkgSSBMZWFybmVkIHRvIFF1aXQgQmVsaWV2aW5nIE15IFByZWp1ZGljZXMKCkluIG15IFtsYXN0CmFydGljbGVdKGh0dHBzOi8vd3d3LmluY2l0ZWRlY2lzaW9udGVjaC5jb20vdHV0b3JpYWxzL0JheWVzX01hbGVfb3JfRmVtYWxlXzEubmIuaHRtbCkKb24gdGhpcyB0b3BpYywgSSBzaG93ZWQgdGhhdCBjb25zaWRlcmluZyBiYWNrZ3JvdW5kIGluZm9ybWF0aW9uIGNhbiBwbGF5IGEKc2lnbmlmaWNhbnQgcm9sZSBpbiBoZWxwaW5nIHVzIG1ha2UgbGVzcyBiaWFzZWQganVkZ21lbnRzLiBXaGF0IEkgaG9wZSB0byBzaG93Cm5vdyBpcyB0aGF0IHdoaWxlIHdlIGxlYXJuIGJ5IHVwZGF0aW5nIHRoZSBpbmZvcm1hdGlvbiB3ZSBoYXZlIHRocm91Z2gKZXhwZXJpZW5jZSwgbGltaXRlZCBleHBlcmllbmNlcyBjYW4gb2Z0ZW4gbGVhZCB0byBwcmVqdWRpY2VzIGFib3V0IHRoZSB3YXkgd2UKaW50ZXJwcmV0IHRoZSB3b3JsZDsgYnV0IGV2ZW4gYnJvYWQgYW5kIGRlZXAgZXhwZXJpZW5jZSBzaG91bGQgcmFyZWx5IGxlYWQgdXMgdG8KY2VydGFpbiBjb25jbHVzaW9ucy4KClRvIGdldCBzdGFydGVkLCBpbWFnaW5lIHBsYXlpbmcgYSBnYW1lIGluIHdoaWNoIHNvbWVvbmUgYXNrcyB5b3UgdG8gaW5mZXIgdGhlCm51bWJlciBvZiBzaWRlcyBvZiBhIGRpZSBiYXNlZCBvbiB0aGUgZmFjZSBudW1iZXJzIHRoYXQgc2hvdyB1cCBpbiByZXBlYXRlZAp0aHJvd3Mgb2YgdGhlIGRpZS4gVGhlIG9ubHkgaW5mb3JtYXRpb24geW91IGFyZSBnaXZlbiBiZWZvcmVoYW5kIGlzIHRoYXQgdGhlCmFjdHVhbCBkaWUgd2lsbCBiZSBzZWxlY3RlZCBmcm9tIGEgc2V0IG9mIHNldmVuIGRpZSBoYXZpbmcgdGhlc2UgbnVtYmVyIG9mCmZhY2VzOiAoNCwgNiwgOCwgMTAsIDEyLCAxNSwgMTgpLiBBc3N1bWluZyB5b3UgY2FuIHRydXN0IHRoZSBwZXJzb24gd2hvIHJlcG9ydHMKdGhlIG91dGNvbWUgb24gZWFjaCB0aHJvdywgYWZ0ZXIgaG93IG1hbnkgcm9sbHMgb2YgdGhlIGRpZSB3aWxsIHlvdSBiZSB3aWxsaW5nCnRvIHNwZWNpZnkgd2hpY2ggZGllIHdhcyBjaG9zZW4/CgpMZXQncyB1c2UgdGhlIFIgcHJvZ3JhbW1pbmcgbGFuZ3VhZ2UgdG8gaGVscCB1cyB0aGluayB0aHJvdWdoIHRoZSBwcm9ibGVtLiBTdGFydApieSBzcGVjaWZ5aW5nIHRoZSBzZXQgb2YgdGhlIGRpZSBwb3NzaWJpbGl0aWVzIHN1Y2ggdGhhdCBlYWNoIG51bWJlciByZXByZXNlbnRzCnRoZSBudW1iZXIgb2Ygc2lkZXMgb2YgYSBnaXZlbiBkaWUuIChZb3UgbWlnaHQgYWxzbyB3YW50IHRvIHJlZmVyIHRvIG15CltwcmV2aW91cwphcnRpY2xlXShodHRwczovL3d3dy5pbmNpdGVkZWNpc2lvbnRlY2guY29tL3R1dG9yaWFscy9CYXllc19NYWxlX29yX0ZlbWFsZV8xLm5iLmh0bWwpCm9uIEJheWVzaWFuIGFuYWx5c2lzIHRvIGZhbWlsaWFyaXplIHlvdXJzZWxmIHdpdGggc29tZSBvZiB0aGUgdGVybWlub2xvZ3kgdGhhdApmb2xsb3dzLikKCltOb3RlOiBJZiB5b3UncmUgbm90IGludG8gcHJvZ3JhbW1pbmcsIHlvdSBjYW4gc2tpbSBwYXN0IHRoZSBjcnVuY2h5IGdlZWt5IGJpdHMgYW5kCnN0aWxsIGdldCB0aGUgZmxhdm9yZnVsIGNyZWFteSBub3VnYXQgaW5zaWRlLiBCdXQgeW91IGhhdmUgdG8gcHJhY3RpY2Ugc2VsZgpjb250cm9sIHRvIGdldCB0aGUgZGVsYXllZCBzYXRpc2ZhY3Rpb24gYmVuZWF0aCB0aGF0IGxheWVyLiBZb3UgY2FuIGRvIGl0IV0KCmBgYHtyfQpkaWUgPC0gYyg0LCA2LCA4LCAxMCwgMTIsIDE1LCAxOCkKbi5kaWUgPC0gbGVuZ3RoKGRpZSkKYGBgCgpUaGUgZ2FtZSBtYXN0ZXIgc2VsZWN0cyBhIGRpZSB0byBiZSBpbiBwbGF5IGJ1dCBrZWVwcyB0aGUgY2hvaWNlIGhpZGRlbiBmcm9tCnVzLiBBcyBJJ2xsIGV4cGxhaW4gaW4gbW9yZSBkZXRhaWwgbGF0ZXIsIHRoZSBnYW1lIG1hc3RlciBpcyB0aGUgYWdncmVnYXRlCm1hcmtldCBvciBuYXR1cmUsIGFuZCBieSAiaW4gcGxheSwiIEkgbWVhbiB0aGF0IHNvbWUgY29uZGl0aW9uIHRoYXQgd2UgY3VycmVudGx5CmNhcmUgYWJvdXQgaW4gdGhhdCBlbnZpcm9ubWVudCBpcyB0aGUgY2FzZS4KCmBgYHtyfQpkaWUuY2hvaWNlIDwtIDEyCmBgYAoKR2l2ZW4gdGhhdCB3ZSBhcmUgdGFza2VkIHRvIGluZmVyIHdoaWNoIGRpZSBpcyBjaG9zZW4sIGFuZCB3ZSBkb24ndCBrbm93IGFueQpvdGhlciBpbmZvcm1hdGlvbiBvdGhlciB0aGFuIHRoZSBudW1iZXIgb2YgZGllIGluIHRoZSBzZXQgKGJlZm9yZSB3ZSBvYnNlcnZlIHRoZQpvdXRjb21lIG9mIGFueSBkaWUgdG9zcyksIGFueSBpbml0aWFsIGd1ZXNzIGlzIGFzIGdvb2QgYXMgdGhlIG90aGVyLgpGdXJ0aGVybW9yZSwgd2UgY2FuIGRlc2NyaWJlIG91ciBpbmNsaW5hdGlvbiB0byBiZWxpZXZlIG9uZSBzaXR1YXRpb24gaXMgdGhlCmNhc2UgdG8gdGhlIGV4Y2x1c2lvbiBvZiBhbnkgb3RoZXJzIHdpdGggYSBudW1iZXIgYmV0d2VlbiAwIGFuZCAxLCB3aGVyZSAwIG1lYW5zCnRoYXQgd2UgZG9uJ3QgYmVsaWV2ZSBhdCBhbGwsIGFuZCAxIG1lYW5zIHRoYXQgd2UgY2VydGFpbmx5IGJlbGlldmUgd2l0aCBubwpyZXNlcnZhdGlvbnMuIFdlJ2xsIGNhbGwgdGhpcyBpbml0aWFsIGd1ZXNzIG91ciBwcmlvciBwcm9iYWJpbGl0eSAob3Igb3VyCmluaXRpYWwgZGVncmVlIG9mIGJlbGllZikgdGhhdCBhbnkgb25lIG9mIHRoZSBkaWNlIGlzIGluIHBsYXkuIFdoYXQgbnVtYmVyCnNob3VsZCB3ZSBhc3NpZ24gZm9yIGVhY2ggb2YgdGhlIHBvc3NpYmlsaXRpZXM/IFdlbGwsIHNpbmNlIHdlIGtub3cgdGhhdCBhbnkKZ3Vlc3MgaXMgYXMgZ29vZCBhcyBhbnkgb3RoZXIsIHRoYXQgdGhlIHBvc3NpYmlsaXRpZXMgYXJlIG11dHVhbGx5IGV4Y2x1c2l2ZSwKYnV0IHRoYXQgYW55IG9uZSBvZiB0aGUgcG9zc2liaWl0aWVzIHdlJ3ZlIGlkZW50aWZpZWQgd2lsbCBjZXJ0YWlubHkgYmUgdGhlCmNhc2UsIGEgbGl0dGxlIHRob3VnaHQgc2hvdWxkIHRlbGwgdXMgdGhhdCBgMS8obnVtYmVyIG9mIGRpY2UpYCBzaG91bGQgYmUgdGhlCnByaW9yIHByb2JhYmlsaXR5IHRoYXQgd2UgYXNzaWduIHRvIGFsbCB0aGUgcG9zc2liaWxpdGllcy4uLmF0IGxlYXN0IGluaXRpYWxseS4KKFNvbWUgcmVhbGx5IHNtYXJ0IG1hdGhlbWF0aWNpYW5zIGRlc2NyaWJlZCB0aGlzIGFzIHRoZSBbcHJpbmNpcGxlIG9mCmluc3VmZmljaWVudCByZWFzb25dKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL1ByaW5jaXBsZV9vZl9pbmRpZmZlcmVuY2UpKQpBcyB3ZSBwcm9jZWVkLCB3ZSdsbCBkaXNjdXNzIGhvdyB0byB1cGRhdGUgdGhlc2UgcHJvYmFiaWxpdGllcyBhcyB3ZSBnZXQgbW9yZQppbmZvcm1hdGlvbiBhcyB3ZSBhZ2dyZWdhdGUgdGhlIG91dGNvbWVzIG9mIHRoZSB0b3NzIG9mIHRoZSBkaWUuIEluIFIsIHdlIGFwcGx5CnRoaXMgcHJvYmFiaWxpdHkgYWNyb3NzIGEgdmVjdG9yIHdpdGggYSBzaXplIGVxdWFsIHRvIHRoZSBzZXQgb2YgZGllCnBvc3NpYmlsaXRpZXMuCgpgYGB7cn0KaW5pdGlhbC5wcm9iIDwtIDEvbi5kaWUKaW5pdGlhbC5wcm9icyA8LSByZXAoaW5pdGlhbC5wcm9iLCBuLmRpZSkKYGBgCgpgYGB7ciBlY2hvPUZBTFNFfQpwcmludChpbml0aWFsLnByb2JzKQpgYGAKCkJlZm9yZSB3ZSBzdGFydCB0aGUgcHJvY2VzcyBvZiBvYnNlcnZpbmcgdGhlIGRpZSBvdXRjb21lcyBhcyB0aGV5IGFyZSB0b3NzZWQsCmNyZWF0ZSBhbiBpbmRleCB0byBrZWVwIHRyYWNrIG9mIHRob3NlIHRyaWFscyBhcyB0aGV5IG9jY3VyLiBMZXQncyBzdGFydCB3aXRoIDIwCnRyaWFscy4KCmBgYHtyfQpuLnRyaWFscyA8LSAyMAp0cmlhbHMgPC0gMDpuLnRyaWFscwpgYGAKCkluaXRpYWxpemUgdGhlIHRyaWFscyBvdXRjb21lIHZlY3Rvci4gVGhlIGZpcnN0IHRyaWFsIG91dGNvbWUgKGkuZS4sIHRyaWFsPTApCndpbGwgYmUgTkEgYmVjYXVzZSB3ZSBkb24ndCBjb25kdWN0IGEgdHJpYWwgdW50aWwgdHJpYWw9MS4KCmBgYHtyfQp0cmlhbC5vdXRjb21lcyA8LSByZXAoMCwgbi50cmlhbHMrMSkKdHJpYWwub3V0Y29tZXNbMV0gPC0gTkEKYGBgCgpFc3RhYmxpc2ggdGhlIHByb2JhYmlsaXR5IHRoYXQgYW55IGZhY2Ugb2YgYSBnaXZlbiBkaWUgb2NjdXJzLCBhc3N1bWluZwp0aGF0IGVhY2ggZGllIGlzIGZhaXIuIEZvciBhbnkgZGllIHRoYXQgY291bGQgYmUgaW4gcGxheSwgdGhlCnByb2JhYmlsaXR5IHRoYXQgYW55IG9mIGl0cyBzaWRlcyBvY2N1cnMgaXMgYDEvKG51bWJlciBvZiBzaWRlcylgLiBUaGlzIGlzCmFub3RoZXIgYXBwbGljYXRpb24gb2YgdGhlIHByaW5jaXBsZSBvZiBpbnN1ZmZpY2llbnQgcmVhc29uLgoKYGBge3J9Cm91dGNvbWUucHJvYiA8LSAxL2RpZQpgYGAKCldlIHdpbGwga2VlcCB0cmFjayBvZiB0aGUgdGhyZWUga2V5IHByb2JhYmlsaXRpZXMgaW4gdGhyZWUgc2VwYXJhdGUgYXJyYXlzIGFzCnRoZSB0cmlhbHMgcmV2ZWFsIHRoZWlyIG91dGNvbWVzLiBXZSBpbml0aWFsaXplIGVhY2ggYXJyYXkgaGVyZS4gVGhlIGZpcnN0IHR3bwphcnJheXMgYXJlIHRoZSBzYW1lIHNpemUgd2l0aCBzaGFwZSBgQXJyYXkobSB4IG4pID0gKG51bWJlciBvZiB0cmlhbHMpIHggKG51bWJlciBvZiBwb3NzaWJpbGl0aWVzKWAuClRoZSB0aGlyZCBpcyBhIHZlY3RvciBvZiBsZW5ndGggYChudW1iZXIgb2YgdHJpYWxzKWAuCgooMSkgVGhpcyBhcnJheSBjb3JyZXNwb25kcyB0byB0aGUgcHJvYmFiaWxpdHkgdHlwaWNhbGx5IGRlbm90ZWQgYXMgcHJvYihldmlkZW5jZQp8IGh5cG90aGVzaXMpIG9yIG91ciAibGlrZWxpaG9vZCIgZnVuY3Rpb24uIEZvciBvdXIgZXhlcmNpc2UsIHdlIGludGVycHJldCB0aGlzCmFzIHRoZSBwcm9iYWJpbGl0eSB0aGF0IHdlIG9ic2VydmUgYSByZXZlYWxlZCBvdXRjb21lIGdpdmVuIHRoYXQgdGhlIGNob2ljZSBvZiBhCmNlcnRhaW4gZGllIGlzIGluIHBsYXkuIFRoZSBmaXJzdCB2YWx1ZXMgYXQgdHJpYWw9MCBhcmUgb3VyCnByaW9yIHByb2JhYmlsaXRpZXMuCgpgYGB7cn0KcHJvYi5vdXRjb21lLmd2LmNob2ljZSA8LQogIGFycmF5KHJlcCgwLCAobi50cmlhbHMrMSkgKiBuLmRpZSksCiAgICAgICAgZGltID0gYygobi50cmlhbHMrMSksIG4uZGllKSkKcHJvYi5vdXRjb21lLmd2LmNob2ljZVsxLCBdIDwtIGluaXRpYWwucHJvYnMKY29sbmFtZXMocHJvYi5vdXRjb21lLmd2LmNob2ljZSkgPC0gZGllCmBgYAoKKDIpIFRoaXMgYXJyYXkgY29ycmVzcG9uZHMgdG8gdGhlIHByb2JhYmlsaXR5IHR5cGljYWxseSBkZW5vdGVkIGFzIApwcm9iKGh5cG90aGVzaXMgfCBldmlkZW5jZSkgb3Igb3VyIHBvc3RlcmlvciBwcm9iYWJpbGl0eS4gRm9yIG91ciBleGVyY2lzZSwgd2UKaW50ZXJwcmV0IHRoaXMgYXMgdGhlIGRlZ3JlZSBvZiBiZWxpZWYgdGhhdCBhIGNlcnRhaW4gZGllIGlzIGluIHBsYXkKZ2l2ZW4gdGhlIGN1bXVsYXRpdmUgb2JzZXJ2YXRpb25zIG9mIG91dGNvbWVzLgoKYGBge3J9CnByb2IuY2hvaWNlLmd2Lm91dGNvbWUgPC0gcHJvYi5vdXRjb21lLmd2LmNob2ljZQpgYGAKCigzKSBUaGlzIHZlY3RvciBjb3JyZXNwb25kcyB0byB0aGUgc3VtIG9mIHByb2JhYmlsaXRpZXMgb2Ygb3V0Y29tZXMgb24gZWFjaAp0cmlhbC4gSW4gZWFjaCB0cmlhbCwgaXQncyB0aGUgbm9ybWFsaXppbmcgY29uc3RhbnQuIFRoZSBmaXJzdCB2YWx1ZSBhdAp0cmlhbD0wIGlzIDEgc2luY2UgaXQgaXMgdGhlIHN1bSBvZiB0aGUgaW5pdGlhbCBwcmlvciBwcm9iYWJpbGl0aWVzIG9yIChudW1iZXIKb2YgcG9zc2liaWxpdGllcykgKiAoMS8obnVtYmVyIG9mIHBvc3NpYmlsaXRpZXMpKS4KCmBgYHtyfQptYXJnaW5hbC5wcm9iIDwtIHNhcHBseSh0cmlhbHMsIGZ1bmN0aW9uKHQpIHN1bShwcm9iLm91dGNvbWUuZ3YuY2hvaWNlW3QrMSwgXSkpCmBgYAoKQmVmb3JlIHdlIHN0YXJ0IG91ciBzaW11bGF0ZWQgZXhwZXJpbWVudCwgbGV0J3MgdGhpbmsgYSBiaXQgbW9yZSBhYm91dCB3aGF0IGVhY2gKb2YgdGhvc2UgdGhyZWUgYXJyYXlzIG1lYW4uIFRoZSBmaXJzdCBhcnJheSB0ZWxscyB1cyBob3cgbGlrZWx5IGl0IGlzIHRoYXQgd2UKc2hvdWxkIG9ic2VydmUgYSBnaXZlbiBvdXRjb21lIGlmIGEgZGllIG9mIGEgY2VydGFpbiBzaXplIGlzIGluIHBsYXkuIElmIHRoZSBkaWUKaXMgZm91ci1zaWRlZCwgdGhlIGxpa2VsaWhvb2QgdGhhdCB3ZSB3b3VsZCBvYnNlcnZlIGEgMSwgMiwgMywgb3IgNCBpcyAxLzQ7Cmxpa2V3aXNlLCBmb3IgYSAxMi1zaWRlZCBkaWUsIGFueSBvbmUgb2YgdGhlIGZhY2VzIHdvdWxkIGhhdmUgYSBsaWtlbGlob29kIG9mCjEvMTIuIFRoZSBzZWNvbmQgYXJyYXkga2VlcHMgdHJhY2sgb2Ygb3VyIGRlZ3JlZXMgb2YgYmVsaWVmIHRoYXQgb25lIG9mIHRoZSBkaWUKdHlwZXMgd2FzIHNlbGVjdGVkIGZvciBwbGF5LiBUaGUgZmlyc3Qgc2V0IG9mIHZhbHVlcyBpbiB0aGlzIGFycmF5IHdpbGwgYmUgb3VyCnByaW9yIHByb2JhYmlsaXRpZXMsIHdoaWNoIGFyZSBhbGwgZXF1YWwsIGJ1dCB0aGV5IHdpbGwgY2hhbmdlIGFzIHdlIG9ic2VydmUgdGhlCm91dGNvbWVzIG9mIHRoZSB0b3NzZXMuIEhvcGVmdWxseSwgdGhleSB3aWxsIGRvIHNvIHN1Y2ggdGhhdCBvdXIgY29uZmlkZW5jZSB0aGF0Cm9uZSBkaWUgdHlwZSBpcyB0aGUgY2FzZSBtb3Jlc28gdGhhbiB0aGUgb3RoZXIgcG9zc2liaWxpdGllcy4gRmluYWxseSwgdGhlIHRoaXJkCnZlY3RvciBpcyB0aGUgc3VtIG9mIHRoZSBsaWtlbGlob29kcyBmb3IgYW55IG91dGNvbWUgdGhhdCBvY2N1cnMuIEZvciBleGFtcGxlLAppZiBhIDkgYXBwZWFycyBvbiBhIHRvc3MsIHRoZSBsaWtlbGlob29kIHRoYXQgd2Ugd291bGQgb2JzZXJ2ZSBhIDkgZm9yIGFueSBvZgp0aGUgZ2l2ZW4gZ2VuZXJhdGluZyBwb3NzaWJpbGl0aWVzIGlzLi4uCgpgYGB7cn0KZGYubGlrZWxpaG9vZCA8LSBkYXRhLmZyYW1lKGRpZSwgaWZlbHNlKDk+PWRpZSwgMCwgb3V0Y29tZS5wcm9iKSkKY29sbmFtZXMoZGYubGlrZWxpaG9vZCkgPC0gYygiRGllIFNpZGVzIiwgIkxpa2VsaWhvb2QgR2l2ZW4gOSIpCmRmLmxpa2VsaWhvb2QgJT4lCiAga2FibGUoKSAlPiUKICBrYWJsZV9zdHlsaW5nKGJvb3RzdHJhcF9vcHRpb25zID0gInN0cmlwZWQiLAogICAgICAgICAgICAgICAgZnVsbF93aWR0aCA9IEYpICU+JQogIGNvbHVtbl9zcGVjKDEsIGJvbGQgPSBULCBib3JkZXJfcmlnaHQgPSBULCB3aWR0aCA9ICIyLjI1Y20iKSAlPiUKICBjb2x1bW5fc3BlYygyLCB3aWR0aCA9ICIzLjc1Y20iKQpgYGAKClRoZSBzdW0gb2YgdGhvc2UgdmFsdWVzIGluIHRoZSByaWdodCBjb2x1bW4gd291bGQgYmUgYGBgcgpzdW0oZGYubGlrZWxpaG9vZCRMaWtlbGlob29kKWBgYCBmb3IgdGhlIHRyaWFsIG9uIHdoaWNoIHdlIG1pZ2h0IG9ic2VydmUgdGhhdCA5LgoKUnVuIHRoZSBzaW11bGF0aW9uIGJ5IHN0ZXBwaW5nIGFjcm9zcyB0aGUgdHJpYWxzIGluZGV4LCBzdGFydGluZyBhdCB0aGUgc2Vjb25kCnBvc2l0aW9uIChpLmUuLCB0cmlhbHM9MSkgYnkgc2tpcHBpbmcgdGhlIGluaXRpYWwgcG9zaXRpb24gKGkuZS4sIHRyaWFscz0wKS4KVGhlIHRlcm0gdHJpYWxzWy0xXSBpcyBhIHJlZHVjZWQgZm9ybSBvZiB0aGUgdHJpYWxzIGluZGV4IHRoYXQgcmVtb3ZlcyB0aGUKaW5pdGlhbCBwb3NpdGlvbi4gQXMgd2Ugc3RlcCB0aHJvdWdoIGVhY2ggdHJpYWwsIHdlIHdpbGwgdXBkYXRlIG91ciBwb3N0ZXJpb3IgcHJvYmFiaWxpdHkgYXMKCmBwb3N0ZXJpb3IgPSBwcmV2aW91c19wb3N0ZXJpb3IgKiBvYnNlcnZlZF9vdXRjb21lX2xpa2VsaWhvb2RzIC8gc3VtKG9ic2VydmVkX291dGNvbWVfbGlrZWxpaG9vZClgCgpgYGB7cn0KZm9yICh0IGluIHRyaWFsc1stMV0pIHsKICAjIEdlbmVyYXRlIGEgbmV3IG91dGNvbWUgZm9yIGVhY2ggdHJpYWwgYmFzZWQgb24gdGhlIGRpZSBpbiBwbGF5LgogIHRyaWFsLm91dGNvbWVzW3QrMV0gPC0gcm91bmQocnVuaWYoMSwgMSwgZGllLmNob2ljZSkpCgogICMgQ2FsY3VsYXRlIHRoZSBuZXcgcHJpb3IgcHJvYmFiaWxpdHkgZm9yIGVhY2ggcG9zc2liaWxpdHkgb24gZWFjaCBuZXcgdHJpYWwuCiAgICBwcm9iLm91dGNvbWUuZ3YuY2hvaWNlW3QrMSwgXSA8LQogICAgICAoMSAtICh0cmlhbC5vdXRjb21lc1t0KzFdID4gZGllKSkgKgogICAgICBwcm9iLmNob2ljZS5ndi5vdXRjb21lW3QsIF0gKiBvdXRjb21lLnByb2IKCiAgICAjIENhbGN1bGF0ZSB0aGUgbWFyZ2luYWwgcHJvYmFiaWxpdHkgaW4gZWFjaCB0cmlhbC4KICAgIG1hcmdpbmFsLnByb2JbdCsxXSA8LSBzdW0ocHJvYi5vdXRjb21lLmd2LmNob2ljZVt0KzEsIF0pCgogICAgIyBDYWxjdWxhdGUgdGhlIHVwZGF0ZWQgcG9zdGVyaW9yIGZvciBlYWNoIHBvc3NpYmlsaXR5IG9uIGVhY2ggdHJpYWwuCiAgICBwcm9iLmNob2ljZS5ndi5vdXRjb21lW3QrMSwgXSA8LSBwcm9iLm91dGNvbWUuZ3YuY2hvaWNlW3QrMSwgXSAvCiAgICAgIG1hcmdpbmFsLnByb2JbdCsxXQp9CmBgYAoKYGBge3J9CmRmLnRyaWFsLm91dGNvbWVzIDwtIGRhdGEuZnJhbWUodHJpYWxzLCB0cmlhbC5vdXRjb21lcykKY29sbmFtZXMoZGYudHJpYWwub3V0Y29tZXMpIDwtIGMoInRyaWFsIiwgIm91dGNvbWUiKQpkZi50cmlhbC5vdXRjb21lcyAlPiUKICBrYWJsZShjYXB0aW9uID0gIlRhYmxlIG9mIHRyaWFscyBhbmQgdGhlIGFzc29jaWF0ZWQgb2JzZXJ2ZWQgb3V0Y29tZSBvbiBlYWNoLiIpICU+JQogIGthYmxlX3N0eWxpbmcoYm9vdHN0cmFwX29wdGlvbnMgPSAic3RyaXBlZCIsCiAgICAgICAgICAgICAgICBmdWxsX3dpZHRoID0gRikgJT4lCiAgY29sdW1uX3NwZWMoMSwgYm9sZCA9IFQsIGJvcmRlcl9yaWdodCA9IFQpICU+JQogIGNvbHVtbl9zcGVjKDE6Miwgd2lkdGggPSAiM2NtIikKYGBgCgpOb3csIHJlZGVmaW5lIHRoZSBwcm9iLmNob2ljZS5ndi5vdXRjb21lIGFycmF5IGFzIGEgZGF0YSBmcmFtZSBmb3IgcGxvdHRpbmcgd2l0aApnZ3Bsb3QyLgoKYGBge3J9CmRmLnByb2IuY2hvaWNlLmd2Lm91dGNvbWUgPC0gYXMuZGF0YS5mcmFtZShwcm9iLmNob2ljZS5ndi5vdXRjb21lLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29sbmFtZXMgPSBkaWUpCmBgYAoKVHJhbnNmb3JtIHRoZSBkYXRhIGZyYW1lIGRmLnByb2IuY2hvaWNlLmd2Lm91dGNvbWUgaW50byBhIGRhdGEgZnJhbWUgd2l0aApyZWxhdGlvbmFsIHN0cnVjdHVyZSBzdWNoIHRoYXQgdHJpYWwgaXMgdGhlIGxlZnQgb3V0ZXJtb3N0IGNvbHVtbiwgZGllCnBvc3NpYmlsaXR5IGlzIHRoZSBuZXh0IGNvbHVtbiwgYW5kIHRoZSB2YWx1ZXMgYXJlIHRoZSBmaW5hbCByaWdodCBjb2x1bW4uCgpgYGB7cn0KbWVsdC5wcm9iLmNob2ljZS5ndi5vdXRjb21lIDwtIG1lbHQoY2JpbmQodHJpYWxzLCBkZi5wcm9iLmNob2ljZS5ndi5vdXRjb21lKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaWQudmFycz0idHJpYWxzIikKYGBgCgpQbG90IHRoZSBwcm9iYWJpbGl0aWVzIG9mIHRoZSBkaWUgcG9zc2liaWxpdGllcyBhcyB0aGV5IGV2b2x2ZSBhY3Jvc3MKdGhlIHRyaWFscy4KCmBgYHtyIGZpZy53aWR0aD0xMCwgZmlnLmhlaWdodD00fQoKcGxvdC5wcm9iLnRyaWFscyA8LQogIGdncGxvdChkYXRhID0gbWVsdC5wcm9iLmNob2ljZS5ndi5vdXRjb21lLAogICAgICAgICAgICAgICAgICAgIGFlcyh4ID0gdHJpYWxzLAogICAgICAgICAgICAgICAgICAgICAgICB5ID0gdmFsdWUsCiAgICAgICAgICAgICAgICAgICAgICAgIGdyb3VwID0gdmFyaWFibGUsCiAgICAgICAgICAgICAgICAgICAgICAgIGNvbG91ciA9IHZhcmlhYmxlKSkKcGxvdC5wcm9iLnRyaWFscyA8LSBwbG90LnByb2IudHJpYWxzICsgZ2VvbV9saW5lKCkKcGxvdC5wcm9iLnRyaWFscyA8LSBwbG90LnByb2IudHJpYWxzICsgZ2VvbV9wb2ludCgpCnBsb3QucHJvYi50cmlhbHMgPC0gcGxvdC5wcm9iLnRyaWFscyArIHRoZW1lKGF4aXMudGV4dCA9IGVsZW1lbnRfdGV4dChzaXplPTE0KSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYXhpcy50aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplPTE0LGZhY2U9ImJvbGQiKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdGl0bGUgPSBlbGVtZW50X3RleHQoc2l6ZT0xNixmYWNlPSJib2xkIikpCnBsb3QucHJvYi50cmlhbHMgPC0gcGxvdC5wcm9iLnRyaWFscyArIGxhYnModGl0bGUgPSAiUHJvYmFiaWxpdHkgVGhhdCBhIEdpdmVuIERpZSBpcyBpbiBQbGF5IiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB4ID0gIlRyaWFscyIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgeSA9ICJQcm9iYWJpbGl0eSIKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICApCnBsb3QocGxvdC5wcm9iLnRyaWFscykKYGBgCgpXZSBjYW4gYWxzbyBwcmVzZW50IHRoaXMgYXMgc2xpY2VzIGZyb20gc29tZSBzZWxlY3RlZCBzdGVwcyBhZnRlciB0aGUgZmlyc3QKdHJpYWwgYWNyb3NzIHRoZSBwcm9iYWJpbGl0aWVzIG9mIHRoZSBkaWUgcG9zc2liaWxpdGllcy4gRGV0ZXJtaW5lIHRoZSByYXRpb25hbApvcmRlcmluZyBvZiBlYWNoIHBvc3NpYmlsaXR5IGluIGVhY2ggc2xpY2UuCgpgYGB7ciBmaWcud2lkdGg9MTAsIGZpZy5oZWlnaHQ9NH0KCnN0ZXBzIDwtIDUKY2hlY2suc3RlcHMgPC0gMTpzdGVwcwp0cmlhbC5zbGljZXMgPC0gYygwLCAxLCBjZWlsaW5nKGNoZWNrLnN0ZXBzKm4udHJpYWxzL3N0ZXBzKSkKCnN1Yi5wcm9iLmNob2ljZS5ndi5vdXRjb21lIDwtIG4uZGllICogbGVuZ3RoKHRyaWFsLnNsaWNlcykKbWVsdC5wcm9iLmNob2ljZS5ndi5vdXRjb21lMiA8LQogIG1lbHQucHJvYi5jaG9pY2UuZ3Yub3V0Y29tZVsxOnN1Yi5wcm9iLmNob2ljZS5ndi5vdXRjb21lLCBdCgpmb3IgKHQgaW4gMTpsZW5ndGgodHJpYWwuc2xpY2VzKSkgewptZWx0LnByb2IuY2hvaWNlLmd2Lm91dGNvbWUyWyh0LTEpICogbi5kaWUgKyAoMTpuLmRpZSksIF0gPC0KICBtZWx0LnByb2IuY2hvaWNlLmd2Lm91dGNvbWVbbWVsdC5wcm9iLmNob2ljZS5ndi5vdXRjb21lJHRyaWFscz09dHJpYWwuc2xpY2VzW3RdLCBdCn0KCnBsb3Quc2VsZWN0aW9uIDwtCiAgZ2dwbG90KGRhdGEgPSBtZWx0LnByb2IuY2hvaWNlLmd2Lm91dGNvbWUyLCBhZXMoeCA9IHZhcmlhYmxlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHkgPSB2YWx1ZSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmaWxsID0gdmFyaWFibGUpKQpwbG90LnNlbGVjdGlvbiA8LSBwbG90LnNlbGVjdGlvbiArIGdlb21fYmFyKHN0YXQ9ImlkZW50aXR5IikKcGxvdC5zZWxlY3Rpb24gPC0gcGxvdC5zZWxlY3Rpb24gKyBmYWNldF9ncmlkKC4gfiB0cmlhbHMpCnBsb3Quc2VsZWN0aW9uIDwtIHBsb3Quc2VsZWN0aW9uICsgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoc2l6ZT0xMiksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYXhpcy50ZXh0LnkgPSBlbGVtZW50X3RleHQoc2l6ZT0xNCksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYXhpcy50aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplPTE0LGZhY2U9ImJvbGQiKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0aXRsZSA9IGVsZW1lbnRfdGV4dChzaXplPTE2LGZhY2U9ImJvbGQiKSkKcGxvdC5zZWxlY3Rpb24gPC0gcGxvdC5zZWxlY3Rpb24gKyBsYWJzKHRpdGxlID0gIkRpZSBQcm9iYWJpbGl0eSBhdCBTZWxlY3RlZCBUcmlhbHMiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgeCA9ICJEaWUgUG9zc2liaWxpdGllcyIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB5ID0gIlByb2JhYmlsaXR5IikKcGxvdChwbG90LnNlbGVjdGlvbikKCmBgYAoKIyMgT2JzZXJ2YXRpb25zIGFuZCBDb25jbHVzaW9ucyAoZm9yIGFzIGxvbmcgYXMgdGhleSBtYXR0ZXIpClNvLCB3aGF0IGRvZXMgYWxsIHRoaXMgbWVhbiwgYmVzaWRlIGxlYXJuaW5nIHNvbWUgbmlmdHkgUiBjb2RlPyBIb3cgZG9lcyBpdApyZWxhdGUgdG8gYnVzaW5lc3MgYW5kIG91ciBpbnR1aXRpb24/CgpMZXQncyBzdGFydCB3aXRoIG91ciBwcmlvciBwcm9iYWJpbGl0eS4gSW4gY2FzZXMgaW4gd2hpY2ggd2UgY2FuIGFjY291bnQgZm9yIHRoZQpwb3NzaWJpbGl0aWVzIG9mIG91dGNvbWVzIGFuZCBmb3Igd2hpY2ggd2UgaGF2ZSBubyBvdGhlciBpbmZvcm1hdGlvbiB0aGFuIGp1c3QKdGhlaXIgZGVzaWduYXRpb25zLCB3ZSBzaG91bGQgc2VlIHRoYXQgYW55IG91dGNvbWUgc2hvdWxkIGJlIHRob3VnaHQgb2YgYXMKZXF1YWxseSBsaWtlbHkuIFdoYXQgaWYgd2UgYXJyYW5nZSB0aG9zZSBvdXRjb21lcyBmcm9tIGxlYXN0IGRlc2lyYWJsZSB0byBtb3N0CmRlc2lyYWJsZS4gSW5zdGVhZCBvZiBvdmVyd29ya2luZyBvdXIgYW54aWV0eSBtdXNjbGUgdGhhdCB0aGUgd29yc3QgY2FzZSB3aWxsCm9jY3VyLCB3ZSBjYW4gYWxsZXZpYXRlIHNvbWUgb2Ygb3VyIHdvcnJ5IGJ5IHJlYWxpemluZyB0aGF0IHRoZSBsaWtlbGlob29kIG9mCmJldHRlciBvdXRjb21lcyB3aWxsIG1vc3QgbGlrZWx5IGJlIHRoZSBjYXNlLiBUaGVyZSBpcyBhbiBvYnZlcnNlIHNpZGUgdG8gdGhpcwpvYnNlcnZhdGlvbiwgdGhvdWdoLiBJbnN0ZWFkIG9mIGJ1aWxkaW5nIHVwIG91ciBob3BlcyBhbmQgZHJlYW1zIHRoYXQgdGhlIGJlc3QKY2FzZSB3aWxsIG9jY3VyLCB3ZSBtaWdodCB0ZW1wZXIgdGhhdCBlbnRodXNpYXNtIGEgYml0IGJ5IHJlYWxpemluZyB0aGF0IGEgbGVzcwp0aGFuIGJlc3QgY2FzZSB3aWxsIG1vc3QgbGlrZWx5IGJlIHRoZSBjYXNlLiBGcm9tIHRoZSBvdXRzZXQsIHdlIHNob3VsZCBiZSBsZXNzCmluY2xpbmVkIHRvIGFsbG9jYXRlIGEgYnVsayBvZiBvdXIgcmVzb3VyY2VzIG9uIHRob3NlIGV4dHJlbWUgZW5kcy4uLnVudGlsIHdlCmdldCBtb3JlIGluZm9ybWF0aW9uLiBSZWdhcmRsZXNzLCBhdCB0aGUgYmVnaW5uaW5nLCB3ZSBzaG91bGQgYmUgcmVzZXJ2ZWQgYWJvdXQKbGV0dGluZyBvdXIgZW1vdGlvbnMgZG9taW5hdGUgb3VyIG1pbmRzIG9uIHRoZSBiYXNpcyBvZiB0aGUgcG9zc2liaWxpdHkgb2YKZXh0cmVtZSBvdXRjb21lcy4gV2Ugc2hvdWxkIGFja25vd2xlZGdlIHRoZXkgZXhpc3QsIGJ1dCB3ZSBzaG91bGRuJ3QgcHVyY2hhc2UKd29ycnkgb3IgZXh1YmVyYW5jZSB1bnRpbCBlaXRoZXIgYXJlIHdhcnJhbnRlZC4gVGhlIHdvcmxkIG9mIG91ciBmdXR1cmUgcHJvYmFibHkKbGllcyBjbG9zZSB0byB0aGUgbWVkaWFuLgoKTm93LCBhcyB3ZSBiZWdpbiB0byBzdGVwIHRocm91Z2ggdGhlIHRyaWFscyAoYW5kIHRyaWJ1bGF0aW9ucyBvZiBsaWZlKSwgd2Ugc2VlCnRoYXQgb3VyIGRlZ3JlZXMgb2YgYmVsaWVmIGFib3V0IHRoZSB0eXBlIG9mIGRpZSB0aGF0IGNvdWxkIGJlIGluIHBsYXkgZXZvbHZlcy4KSW4gZmFjdCwgdGhlIHZlcnkgZmlyc3QgdHJpYWwgb2J2aWF0ZXMgdGhlIG5lZWQgZm9yIGh5cG90aGVzaXppbmcgdGhhdCBhIGZvdXIKc2lkZWQgZGllIGNvdWxkIGJlIHRoZSBjYXNlIGJlY2F1c2Ugd2Ugb2JzZXJ2ZWQgYSA2LiBVbmRlciBubyBjaXJjdW1zdGFuY2VzIGNhbgphIGZvdXItc2lkZWQgZGllIHByb2R1Y2UgYSA2OyB0aHVzLCBvdXIgZGVncmVlIG9mIGJlbGllZiBpbiA0IGNvbGxhcHNlcyB0byAwLgpUaGlzIGlzIGFuIGV4YW1wbGUgb2YgZGlzY29uZmlybWluZyBldmlkZW5jZSwgYSBoYWxsbWFyayBvZiB0aGUgc2NpZW50aWZpYwpwcm9jZXNzLCB0aGUgcHVycG9zZSBvZiB3aGljaCBpcyB0byBmaW5kIHJlYXNvbnMgKipub3QqKiB0byBiZWxpZXZlIHRoYXQgYQpwb3N0dWxhdGVkIGJlbGllZiBpcyB0cnVlLiBJbiB0aGlzIGNhc2UsIHdlIGZvdW5kIG9uZS4gVGhlcmUgaXMgbm8gbW9yZSBuZWVkIHRvCndhc3RlIHRpbWUgcGxhbm5pbmcgb24gNC1zaWRlZCBkaWUgYmVpbmcgdGhlIGNhdXNlIG9mIHRoZSBldmVudHMgd2Ugb2JzZXJ2ZS4KCkFzIG91ciBldmlkZW5jZSBhY2N1bXVsYXRlcywgdHdvIHBvc3NpYmlsaXRpZXMgKHRoZSA2LSBvciA4LXNpZGVkIGRpZSkgaGVscCB1cwp0byBpbmNyZWFzZSBvdXIgaW5jbGluYXRpb24gdGhhdCBlaXRoZXIgaXMgdGhlIGNhc2UgYXMgdGhlIG90aGVyIHBvc3NpYmlsaXRpZXMKbWVsdCBhd2F5LiBUaGlzIGlzIGhvdyBleHBlcmllbmNlIGhlbHBzIHVzIHRvIGxlYXJuLiBBcyB3ZSBzZWUgbW9yZSBhbmQgbW9yZQpleGFtcGxlcyBvZiBldmlkZW5jZSBmb3IgdGhlIHBvc3NpYmlsaXR5IG9mIGEgZ2l2ZW4gY2F1c2FsIGV4cGxhbmF0aW9uLCB3ZQppbmNyZWFzaW5nbHkgYWNjZXB0IHRoZSBkb21pbmF0ZSBjYXNlIHRvIGJlIHRydWUuIEJ1dCBhcyB5b3Ugc2VlLCB0aGlzIG9ubHkKd29ya3MgZm9yIGEgc2hvcnQgd2hpbGUuIFRoZXJlIGlzIGEgZGFyayBzaWRlIHRvIGV4cGVyaWVuY2UsIGFuZCBpdCdzIGNhbGxlZAoqKnByZWp1ZGljZSoqLiBJbiBvdXIgZXhwZXJpbWVudCBoZXJlLCBieSB0aGUgZW5kIG9mIDggdHJpYWxzLCB3ZSB3b3VsZCBoYXZlCnJlYWNoZWQgYSBzdHJvbmcgZGVncmVlIG9mIGJlbGllZiB0aGF0IHRoZSA2LXNpZGVkIGRpZSBpcyB0aGUgY2F1c2FsIG1lY2hhbmlzbQpmb3IgdGhlIGV2ZW50cyB3ZSBvYnNlcnZlLiBCdXQgdGhlbiBhIDkgc2hvd3MgdXAsIGFuZCBvdXIgbm93IHBvc3NpYmx5IGNoZXJpc2hlZApiZWxpZWYgKHRoZSA2LXNpZGVkIGRpZSBleHBsYWlucyBteSB3b3JsZCkgaXMgZGFzaGVkIGFnYWluc3QgdGhlIHJvY2tzLiBUaGluawphYm91dCB0aGlzIGZvciBhIG1vbWVudC4gVGhlcmUgYXJlIG1hbnkgZXZlbnRzIHRoYXQgZG9uJ3Qgb2NjdXIgd2l0aCBhIGxhcmdlCmZyZXF1ZW5jeSwgYW5kIHdlIG1heSBiZSBmb3J0dW5hdGUgb3IgdW5mb3J0dW5hdGUgdG8gb2JzZXJ2ZSBhcyBtYW55IGFzIDggb3IgMTAKb2YgdGhlbSBpbiBvdXIgbGlmZXRpbWVzLiBCdXQgaG93IG1hbnkgdGltZXMgaGF2ZSB3ZSBlaXRoZXIgZXhob3J0ZWQgb3Igd2FybmVkCm90aGVycyBvbiBvdXIgInZhc3QgZXhwZXJpZW5jZSIgdGhhdCB3ZSBrbm93IHRoZSB1bmRlcmx5aW5nIGNhdXNlcyBmb3Igd2hhdCB0aGV5Cm1heSBiZSBmYWNpbmc/IExldCdzIGFkbWl0IGl0IGhlcmU6IGFnZSBjb25mZXJzIG9uIHVzIGEgY2VydGFpbiBhbW91bnQgb2YKZXhwZXJpZW5jZSwgc29tZSBvZiB3aGljaCBtYXkgYmUgc2ltcGx5IHByZWp1ZGljZS4KCkFzIG91ciBleHBlcmltZW50IHByb2NlZWRzLCB3ZSBleHBlcmllbmNlIGFnYWluIHRoYXQgYW5vdGhlciBleHBsYW5hdGlvbiBmb3IgdGhlCndvcmxkIGluY3JlYXNlcyBpdHMgZG9taW5hbmNlLCBvbmx5IHRvIGJlIHNob3QgZG93biBsaWtlIG91ciBsYXN0IHN0cm9uZ2x5IGhlbGQKYmVsaWVmLiBXaWxsIHRoZXJlIGV2ZXIgYmUgYW55IHJlbGllZiB0byB0aGlzIGN5Y2xlIG9mIGxlYXJuaW5nIGFuZCByZWxlYXJuaW5nPwpNYXliZS4gTWF5YmUgbm90LiBOb3RpY2Ugd2hhdCBoYXBwZW5zIHdpdGggdGhlIDE1LXNpZGVkIGRpZS4gSXQncyBhc3NvY2lhdGVkCmRlZ3JlZSBvZiBiZWxpZWYgZGVjbGluZXMgYWxtb3N0IHRvIDAsIHJlc3VycmVjdHMgYWdhaW4gb24gdHJpYWwgIzksIGRlY2xpbmVzCmFsbW9zdCB0byAwLCBhbmQgdGhlbiBwdWxscyBhIHRoaXJkIGFwcGVhcmFuY2Ugb24gdHJpYWwgIzE2LCBidXQgaXQgbmV2ZXIgcXVpdGUKbWFrZXMgaXQgYmFjayBkb3duIHRvIDAgZXZlbiBhcyB0aGUgYXNzb2NpYXRlZCBkZWdyZWUgb2YgYmVsaWVmIGluIHRoZSAxMi1zaWRlZApkaWUgYXBwcm9hY2hlcyAwLjk5LiBUaGlzIGlzIHRoZSBleGFjdCBzYW1lIHBhdHRlcm4gd2Ugb2JzZXJ2ZWQgaW4gdGhlIDEwLSBhbmQKMTItc2lkZWQgZGllLCBsZWFkaW5nIHVzIHRvIGJlbGlldmUgdGhhdCBlaXRoZXIgbW9zdCBsaWtlbHkgd291bGQgbm90IGV4cGxhaW4Kb3VyIHdvcmxkIHVudGlsIHRoYXQgcGVza3kgZGlzY29uZmlybWluZyBldmlkZW5jZSBzaG93ZWQgdXAgdG8gZGlzYWJ1c2UgdXMgb2YKb3VyIGJlbGllZi4gU28sIGFmdGVyIDIwIHRyaWFscywgc2hvdWxkIHdlIGJlIGNvbWZvcnRhYmxlIGJlbGlldmluZyB0aGF0IHRoZQoxMi1zaWRlZCBkaWUgaXMsIGluIGZhY3QsIHRoZSBnYW1lIHBpZWNlIGNob3NlbiBieSB0aGUgZ2FtZSBtYXN0ZXI/IEEgMC45OQpkZWdyZWUgb2YgYmVsaWVmLCBhcyBtb3N0IG9mIGFsbCB0aGUgb3RoZXIgZXhwbGFuYXRpb25zIGhhdmUgYmVlbiBkaXNjb25maXJtZWQsCmVtcGlyaWNhbGx5IGFuZCByYXRpb25hbGx5IGRpc2NhcmRlZCwgaXMgYSBnb29kIHJlYXNvbiB0byBiZXQgb24gdGhhdCBpZGVhLgoKQnV0IGRvbid0IGdldCBjb2NreSwga2lkLiBBbHRob3VnaCB3ZSBrbm93IGluIHRoZSBvbW5pc2NpZW50IHNldHVwIHRoYXQgdGhlCjEyLXNpZGVkIGRpZSBpcyB0aGUgb25lIGluIHBsYXksIHdlIGRvbid0IGtub3cgdGhhdCBpbiBvdXIgc2ltdWxhdGVkIHdvcmxkLiBUaGUKMTUtIGFuZCAxOC1zaWRlZCBkaWUgaGF2ZSBub3QgeWV0IHNob3duIHVzIGV2aWRlbmNlIG9mIHRoZWlyIGV4aXN0ZW5jZS4gSW4gb3VyCnNpbXVsYXRlZCB3b3JsZCwgd2UgbWlnaHQgdGhpbmsgdGhpcyBjb3VsZCBiZSBkdWUgdG8gdGhlIHJlbGF0aXZlbHkgbG93CnByb2JhYmlsaXR5IG9mIGFueSBudW1iZXIgZ3JlYXRlciB0aGFuIDEyIGFwcGVhcmluZy4gQW5kIHNvLCBhbm90aGVyIGV4cGxhbmF0aW9uCmFsd2F5cyBsdXJrcywgYXdhaXRpbmcgdG8gZGlzY29uZmlybSBvdXIgY3VycmVudGx5IGhlbGQgY2hlcmlzaGVkIGJlbGllZiBpbgoxMi1zaWRlZCBkaWUuCgpFeHBlcmllbmNlIGlzIGNlcnRhaW5seSBhIGdvb2QgdGhpbmcuIEl0J3MgaG93IHdlIGxlYXJuLCBidXQgaXQgZG9lc24ndCBhY2NvdW50CmZvciBldmVyeXRoaW5nLiBTaW1wbGUgc2ltdWxhdGlvbnMgbGlrZSB0aGUgcHJlY2VkaW5nIG9uZSBjYW4gaGVscCB1cyB1bmRlcnN0YW5kCnRoYXQgYW5kIGhlbHAgdXMgdGhpbmsgYWJvdXQgaG93IHdlIGNhbiBhZGRyZXNzIHRoYXQgY29uc3RyYWludC4gQnV0IGl0IHdvbid0IGJlCmVhc3kuIExpZmUgbmV2ZXIgaXMuIENvbnNpZGVyIHRoZSBmb2xsb3dpbmcgdHdvIGNvbnVtZHJ1bXMuCgoxLiAqKldlIHByb2JhYmx5IHdvbid0IGJlIGFibGUgdG8gYWNjb3VudCBmb3IgYWxsIHRoZSBwb3NzaWJsZSBleHBsYW5hdGlvbnMgZm9yCnRoZSBlZmZlY3RzIHdlIG9ic2VydmUuKiogT25lIG9mIHRoZSBiZXN0IHdheXMgdG8gYWRkcmVzcyB0aGlzIGlzIHRvIGRyYXcgb24gYQpkaXZlcnNpdHkgb2YgZXhwZXJpZW5jZXMgYW5kIHBlcnNwZWN0aXZlcyBvZiBvdGhlcnMgdG8gY29uc2lkZXIgd2hhdCB3ZSBhbG9uZQpuZXZlciBtaWdodC4KMi4gKipJdCdzIHBvc3NpYmxlIHRoYXQgd2hhdCB3ZSBzZWUgaXNuJ3Qgd2hhdCB3ZSBzaG91bGQgYmVsaWV2ZS4qKiBFeWUgd2l0bmVzcwphY2NvdW50cyBhcmUgbm90b3Jpb3VzbHkgdW5yZWxpYWJsZSwgZXNwZWNpYWxseSBpbiBoaWdoIHN0cmVzcywgaGlnaCB0ZW1wbwplbnZpcm9ubWVudHMuIE91ciBtZWFzdXJlbWVudCBkZXZpY2VzIGFyZSBzb21ldGltZXMgbWlzY2FsaWJyYXRlZCBvciBpbmhlcmVudGx5CmluYWNjdXJhdGUuIFN1cHBvc2UsIGZvciBleGFtcGxlLCB0aGF0IGVhY2ggdGltZSB3ZSByZWFkIHRoZSBudW1iZXIgb24gdGhlCnRvc3NlZCBkaWUsIHRoZXJlIGlzIGEgaW50cmluc2ljIHRlbmRlbmN5IHRvIG1pc3JlYWQgdGhhdCBvdXRjb21lLCBtYXliZSBkdWUgdG8KZmF0aWd1ZSwgYWlsaW5nIGV5ZXMsIG9yIHRoZSBsaWtlLiBPciBzdXBwb3NlIHdlIGFyZSB1c2luZyBhIG1hY2hpbmUgdGhhdCByZWFkcwphIDkgYXMgYSA0IGV2ZXJ5IDEwdGggdGltZS4gU29tZXRpbWVzIHRoZSBwb3NzaWJpbGl0aWVzIGNhbiBiZSBtaXNsYWJlbGVkLAplaXRoZXIgYnkgYWNjaWRlbnQsIGxhemluZXNzLCBvciBpbnRlbnQuIEluIHRoZSBjYXNlIHRoYXQgbGFjayBvZiBhY2N1cmFjeSBpcyBhCnNpZ25pZmljYW50IGNvbmNlcm4sIHdlIHBvc3NpYmx5IHdpbGwgbmV2ZXIgYmUgYWJsZSB0byBkaXNjb25maXJtIGFueSBoeXBvdGhlc2lzCmFsdG9nZXRoZXIuCgpCYWNrIHRvIFtJbmNpdGUhIERlY2lzaW9uIFRlY2hub2xvZ2llc10oaHR0cDovL3d3dy5pbmNpdGVkZWNpc2lvbnRlY2guY29tKS4K

Comments

Collapse Expand Comments (0)
You don't have permission to post comments.

Oracle Crystal Ball Spreadsheet Functions For Use in Microsoft Excel Models

Oracle Crystal Ball has a complete set of functions that allows a modeler to extract information from both inputs (assumptions) and outputs (forecast). Used the right way, these special Crystal Ball functions can enable a whole new level of analytics that can feed other models (or subcomponents of the major model).

Understanding these is a must for anybody who is looking to use the developer kit.

Why are analytics so important for the virtual organization? Read these quotes.

Jun 26 2013
6
0

Since the mid-1990s academics and business leaders have been striving to focus their businesses on what is profitable and either partnering or outsourcing the rest. I have assembled a long list of quotes that define what a virtual organization is and why it's different than conventional organizations. The point of looking at these quotes is to demonstrate that none of these models or definitions can adequately be achieved without some heavy analytics and integration of both IT (the wire, the boxes and now the cloud's virtual machines) and IS - Information Systems (Applications) with other stakeholder systems and processes. Up till recently it could be argued that these things can and could be done because we had the technology. But the reality is, unless you were an Amazon, e-Bay or Dell, most firms did not necessarily have the money or the know-how to invest in these types of inovations.

With the proliferation of cloud services, we are finding new and cheaper ways to do things that put these strategies in the reach of more managers and smaller organizations. Everything is game... even the phone system can be handled by the cloud. Ok, I digress, Check out the following quotes and imagine being able to pull these off without analytics.

The next posts will treat some of the tools and technologies that are available to make these business strategies viable.

Multi-Dimensional Portfolio Optimization with @RISK

Jun 28 2012
16
0

Many speak of organizational alignment, but how many tell you how to do it? Others present only the financial aspects of portfolio optimization but abstract from how this enables the organization to meets its business objectives.  We are going to present a practical method that enables organizations to quickly build and optimize a portfolio of initiatives based on multiple quantitative and qualitative dimensions: Revenue Potential, Value of Information, Financial & Operational Viability and Strategic Fit. 
                  
This webinar is going to present these approaches and how they can be combined to improve both tactical and strategic decision making. We will also cover how this approach can dramatically improve organizational focus and overall business performance.

We will discuss these topics as well as present practical models and applications using @RISK.

Reducing Project Costs and Risks with Oracle Primavera Risk Analysis

.It is a well-known fact that many projects fail to meet some or all of their objectives because some risks were either: underestimated, not quantified or unaccounted for. It is the objective of every project manager and risk analysis to ensure that the project that is delivered is the one that was expected. With the right know-how and the right tools, this can easily be achieved on projects of almost any size. We are going to present a quick primer on project risk analysis and how it can positively impact the bottom line. We are also going to show you how Primavera Risk Analysis can quickly identify risks and performance drivers that if managed correctly will enable organizations to meet or exceed project delivery expectations.

.

 

Modeling Time-Series Forecasts with @RISK


Making decisions for the future is becoming harder and harder because of the ever increasing sources and rate of uncertainty that can impact the final outcome of a project or investment. Several tools have proven instrumental in assisting managers and decision makers tackle this: Time Series Forecasting, Judgmental Forecasting and Simulation.  

This webinar is going to present these approaches and how they can be combined to improve both tactical and strategic decision making. We will also cover the role of analytics in the organization and how it has evolved over time to give participants strategies to mobilize analytics talent within the firm.  

We will discuss these topics as well as present practical models and applications using @RISK.

The Need for Speed: A performance comparison of Crystal Ball, ModelRisk, @RISK and Risk Solver


Need for SpeedA detailed comparison of the top Monte-Carlo Simulation Tools for Microsoft Excel

There are very few performance comparisons available when considering the acquisition of an Excel-based Monte Carlo solution. It is with this in mind and a bit of intellectual curiosity that we decided to evaluate Oracle Crystal Ball, Palisade @Risk, Vose ModelRisk and Frontline Risk Solver in terms of speed, accuracy and precision. We ran over 20 individual tests and 64 million trials to prepare comprehensive comparison of the top Monte-Carlo Tools.

 

Excel Simulation Show-Down Part 3: Correlating Distributions

Escel Simulation Showdown Part 3: Correlating DistributionsModeling in Excel or with any other tool for that matter is defined as the visual and/or mathematical representation of a set of relationships. Correlation is about defining the strength of a relationship. Between a model and correlation analysis, we are able to come much closer in replicating the true behavior and potential outcomes of the problem / question we are analyzing. Correlation is the bread and butter of any serious analyst seeking to analyze risk or gain insight into the future.

Given that correlation has such a big impact on the answers and analysis we are conducting, it therefore makes a lot of sense to cover how to apply correlation in the various simulation tools. Correlation is also a key tenement of time series forecasting…but that is another story.

In this article, we are going to build a simple correlated returns model using our usual suspects (Oracle Crystal Ball, Palisade @RISK , Vose ModelRisk and RiskSolver). The objective of the correlated returns model is to take into account the relationship (correlation) of how the selected asset classes move together. Does asset B go up or down when asset A goes up – and by how much? At the end of the day, correlating variables ensures your model will behave correctly and within the realm of the possible.

Copulas Vs. Correlation

Copulas and Rank Order Correlation are two ways to model and/or explain the dependence between 2 or more variables. Historically used in biology and epidemiology, copulas have gained acceptance and prominence in the financial services sector.

In this article we are going to untangle what correlation and copulas are and how they relate to each other. In order to prepare a summary overview, I had to read painfully dry material… but the results is a practical guide to understanding copulas and when you should consider them. I lay no claim to being a stats expert or mathematician… just a risk analysis professional. So my approach to this will be pragmatic. Tools used for the article and demo models are Oracle Crystal Ball 11.1.2.1. and ModelRisk Industrial 4.0

Excel Simulation Show-Down Part 2: Distribution Fitting

 

One of the cool things about professional Monte-Carlo Simulation tools is that they offer the ability to fit data. Fitting enables a modeler to condensate large data sets into representative distributions by estimating the parameters and shape of the data as well as suggest which distributions (using these estimated parameters) replicates the data set best.

Fitting data is a delicate and very math intensive process, especially when you get into larger data sets. As usual, the presence of automation has made us drop our guard on the seriousness of the process and the implications of a poorly executed fitting process/decision. The other consequence of automating distribution fitting is that the importance of sound judgment when validating and selecting fit recommendations (using the Goodness-of-fit statistics) is forsaken for blind trust in the results of a fitting tool.

Now that I have given you the caveat emptor regarding fitting, we are going to see how each tools offers the support for modelers to make the right decisions. For this reason, we have created a series of videos showing comparing how each tool is used to fit historical data to a model / spreadsheet. Our focus will be on :

The goal of this comparison is to see how each tool handles this critical modeling feature.  We have not concerned ourselves with the relative precision of fitting engines because that would lead us down a rabbit hole very quickly – particularly when you want to be empirically fair.

RESEARCH ARTICLES | RISK + CRYSTAL BALL + ANALYTICS