Monte Carlo Response Surfaces

Using Response Surface Data

Once you download the .mat file associated with a response surface, you will see two variables:

• model
  • matrix of size $N_{vars}\times N_{polynomialterms}$
  • number of columns is equal to the number of input variables $N_{vars}$
  • number of rows is equal to the number of terms in the polynomial response surface $N_{polynomialterms}$
  • variable order is as follows:
    • Mass flowrate ${\dot {m}}$
    • Reaction rate $k(T)$
    • Mixing length $L_{mix}$
    • Measurement location 1 $z_{1}$
    • Measurement location 2 $z_{2}$
    • Measurement location 3 $z_{3}$

• response_surface
  • cell object that results from Matlab's regstats() function
  • polynomial coefficients corresponding to each row of model object are contained in response_surface.beta
  • covariance matrix is stored in response_surface.covb
  • R-squared and adjusted R-squared values stored in response_surface.rsquare and response_surface.adjrsquare, respectively
  • mean square error is contained in response_surface.mse

See the Matlab regstats() help page for more information: http://www.mathworks.com/help/toolbox/stats/regstats.html

Example of how to use this information to compute the value of a response surface is here: EvaluateResponseSurface.m

Response Surface Results

Yp at X1

Quadratic Response, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x1_6dim_2deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 28. 
Degree of response surface is 2.

MSE =			 0.00388068 
MSE DoF = 		 5007 

L-inf norm resid = 	 0.45155983 

R^2 =			 0.91662780 
adjusted R^2 =		 0.91617822 
---------------------------------------------------

Quadratic Response, 2 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x1_2dim_2deg.mat

This is a response surface with the same data, but only regressing on the two visualized variables.

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 2. 
Number of terms in polynomial is 6. 
Degree of response surface is 2.

MSE =			 0.00389885 
MSE DoF = 		 5029 

L-inf norm resid = 	 0.43695247 

R^2 =			 0.91586950 
adjusted R^2 =		 0.91578585 
---------------------------------------------------

Cubic Response, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x1_6dim_3deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 84. 
Degree of response surface is 3.

MSE =			 0.00101396 
MSE DoF = 		 4951 

L-inf norm resid = 	 0.21757339 

R^2 =			 0.97845992 
adjusted R^2 =		 0.97809881 
---------------------------------------------------

Quartic Response, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x1_6dim_4deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 210. 
Degree of response surface is 4.

MSE =			 0.00025835 
MSE DoF = 		 4825 

L-inf norm resid = 	 0.07439991 

R^2 =			 0.99465134 
adjusted R^2 =		 0.99441966 
---------------------------------------------------

Yp at X2

Quadratic Response, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x2_6dim_2deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 28. 
Degree of response surface is 2.

MSE =			 0.02215980 
MSE DoF = 		 5007 

L-inf norm resid = 	 0.47094014 

R^2 =			 0.80184973 
adjusted R^2 =		 0.80078121 
---------------------------------------------------

Quadratic Response, 2 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x2_2dim_2deg.mat

This is a response surface with the same data, but only regressing on the two visualized variables.

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 2. 
Number of terms in polynomial is 6. 
Degree of response surface is 2.

MSE =			 0.02219639 
MSE DoF = 		 5029 

L-inf norm resid = 	 0.43596913 

R^2 =			 0.80065039 
adjusted R^2 =		 0.80045219 
---------------------------------------------------

Cubic Response, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x2_6dim_3deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 84. 
Degree of response surface is 3.

MSE =			 0.01047521 
MSE DoF = 		 4951 

L-inf norm resid = 	 0.33881196 

R^2 =			 0.90737957 
adjusted R^2 =		 0.90582686 
---------------------------------------------------

Quartic Response, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x2_6dim_4deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 210. 
Degree of response surface is 4.

MSE =			 0.00485155 
MSE DoF = 		 4825 

L-inf norm resid = 	 0.22879783 

R^2 =			 0.95819495 
adjusted R^2 =		 0.95638412 
---------------------------------------------------

Yp at X3

Quadratic Response, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x3_6dim_2deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 28. 
Degree of response surface is 2.

MSE =			 0.03700744 
MSE DoF = 		 5007 

L-inf norm resid = 	 0.50180157 

R^2 =			 0.72177521 
adjusted R^2 =		 0.72027490 
---------------------------------------------------

Quadratic Response, 2 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x3_2dim_2deg.mat

This is a response surface with the same data, but only regressing on the two visualized variables.

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 2. 
Number of terms in polynomial is 6. 
Degree of response surface is 2.

MSE =			 0.03703099 
MSE DoF = 		 5029 

L-inf norm resid = 	 0.47547043 

R^2 =			 0.72037494 
adjusted R^2 =		 0.72009692 
---------------------------------------------------

Cubic Response, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x3_6dim_3deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 84. 
Degree of response surface is 3.

MSE =			 0.02076042 
MSE DoF = 		 4951 

L-inf norm resid = 	 0.46211638 

R^2 =			 0.84566717 
adjusted R^2 =		 0.84307989 
---------------------------------------------------

Quartic Response, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_x3_6dim_4deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 210. 
Degree of response surface is 4.

MSE =			 0.01150170 
MSE DoF = 		 4825 

L-inf norm resid = 	 0.30974734 

R^2 =			 0.91667246 
adjusted R^2 =		 0.91306304 
---------------------------------------------------

Yp at exit

Quadratic Surface, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_out_6dim_2deg.mat

A quadratic response surface was computed using all of the information from the Monte Carlo samples. There were 10,000 samples in total.

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 28. 
Degree of response surface is 2.

MSE =			 0.04265621 
MSE DoF = 		 5007 

L-inf norm resid = 	 0.53414457 

R^2 =			 0.68956066 
adjusted R^2 =		 0.68788663 
---------------------------------------------------

Quadratic Surface, 2 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_out_2dim_2deg.mat

The same set of Monte Carlo samples was fit to a quadratic surface, but with 2 variables instead of 6. This results in a response surface that looks similar to the 6-dimensional quadratic response surface:

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

The statistics show that the fit is better for the 2-dimensional surface than for the 6-dimensional surface. This, combined with the fact that he response surfaces look similar, means we can conclude that the additional dimensions are probably independent of the two visualized dimensions, or that they ave a minimal impact on the response.

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 2. 
Number of terms in polynomial is 6. 
Degree of response surface is 2.

MSE =			 0.04267264 
MSE DoF = 		 5029 

L-inf norm resid = 	 0.50344056 

R^2 =			 0.68807653 
adjusted R^2 =		 0.68776641 
---------------------------------------------------

Cubic Surface, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_out_6dim_3deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 84. 
Degree of response surface is 3.

MSE =			 0.02514706 
MSE DoF = 		 4951 

L-inf norm resid = 	 0.51330364 

R^2 =			 0.81903400 
adjusted R^2 =		 0.81600023 
---------------------------------------------------

Quartic Surface, 6 Dimensions

Download the response surface here: http://files.charlesmartinreid.com/ExperimentalDesign/MCResponseSurface_Yp_out_6dim_4deg.mat

The following is a visualization of the response surface (non-visualized dimensions are kept constant at their mean value):

Some key statistics for this response surface are given below:

---------------------------------------------------
Response surface summary of information:
Number of variables in response surface is 6. 
Number of terms in polynomial is 210. 
Degree of response surface is 4.

MSE =			 0.01467238 
MSE DoF = 		 4825 

L-inf norm resid = 	 0.34656843 

R^2 =			 0.89710012 
adjusted R^2 =		 0.89264291 
---------------------------------------------------

It is clear that despite having a high-degree polynomial with a large number (210) of coefficients, the polynomial fit is still quite poor, and increasing the degree of the polynomial does not greatly increase the polynomial's fit to the data.

With the composite design response surface, the (reduced) third degree polynomial fit all of the data points exactly, and yielded 0 mean square error and an r-squared value of 1.0. However, this is because there were only 45 sample points, and almost as many polynomial coefficients - 37.