Breakthrough DoE

Hi my name is Adam. For over 20 years I have been dealing with broadly understood process improvement. I had the opportunity to work under the umbrella of: Six Sigma, Lean Sigma, Continiuous Improvement, KVP (kontinuierliche Verbesserungsprozesse), Kaizen, Factory Excellence. With these different methods, it is like playing football at different clubs, but you always have to kick the ball ;-). This time I will not focus on methodologies but on the selected tool, which is DoE.

Have you been working for months and it actually did not change anything in your professional life? In turn, from time to time an event appears at work that triggers a series of other events and nothing is as it was before?

Such a case was in my professional career with a certain DoE. I started working at the company Macopharama. It was year 2007. And at the very beginning, I could have spoiled something by not knowing the company, processes and people or I could have used a chance (the one and only one) to make a good impression and gain a credit of trust in order to be able to continue further activities.

Undoubtedly, such a breakthrough event for me was the DoE, the description of which can be found later in the article.

It was a battle test, in the reality of the company, a test of the strictly scientific approach of DoE.
The success of the experiment gave a huge credit of trust, which allowed me to do further actions, i.e .:
- Introduction of One Piece Flow.
- SMED - Single Minute Exchange of Die.
- MES - Manufacturing Execution System.

DoE - Design of Experiment, which wrongly can be understood as Experiment Planning. The name can be confusing as it indicates that every experiment needs to be designed (planned), and one might think that there is nothing else to be found besides planning. Design of Experiment is, however, a complete method that includes several variants (from the simplest to the most complicated). In publications about DoE you will surely find information about:

Problem definition.
Selection of an output variable, which can be called Y.
Factors' selection (can also be called X variables or inputs).
Choosing optimization: are we looking for factors that affect the mean, factors that affect the standard deviation or both.
Selection of the factor levels (minimum is 2 levels, e.g. temperature 40 and 60 degrees Celsius).
Selection of an experiment plan.
Analysis of the experiment results.

Introduction to a DoE subjected process.

Before any experiment, it is necessary to know the product manufacturing process, characteristics of the process and the product and machine on which the product is manufactured. Of course, as part of the planning of the experiment, the number of characteristics will be narrowed down and machines will be selected, because it is impossible to check everything in DoE.

My DoE concerned the process of filling blood donation systems with preservative solution. Fig. 1 shows an FQD system with a blood filter used at blood donation points. The filling process was carried out on the filling machines shown in Fig. 2. The choice of the Y seemed obvious due to problems with the stability of the filling process. Historical measurements showed fluctuations in solution volume due to unknown causes.

Diagram — Fig. 1 4-container blood system of FQE type.

So Y is the volume of the solution, but I also took into account the standard deviation of the solution volume or the process capability of the filling process. I have to mention the steps that need to be done before the DoE. Even if the DoE is not a part of the Six Sigma project, there are at least 2 things to check:

The Measurement System Analysis.
Study to assess the initial process capability.

Explanations for p. 1). A scale was used to measure the volume, based on the proportions that 1 ml of the preservative solution had a mass of 1.025 g. For further studies it could be assumed that 1 g = 1 ml. Although the scale turned out to be a good measuring device, as shown in Fig. 3, the analysis of the measuring system procedures caused that one immediately resigned from the so-called taring of the scale. More about MSA and the necessity to do it before experiments in 2 articles:

Explanations for p 2). I wrote “Study to assess the initial process capability”, But I meant any study of an existing process that will show the level of variability of the process broken down by levels. In my case, I did the study I named SPC and described in a separate article. It was very important to me that I could do the analysis up to the level of a single nozzle. Having the data from the SPC, I easily found the Cp (short-term process capacity coefficient), which is presented in Tab. 1. Finding such large differences in the case of seemingly identical machines is a very good prognosis, as it clearly indicates the potential for improvement. If there is a high Cp factor for one machine, then it may be for other machines as well. Now it remained to find the causes, and in my case it was the task for DoE.

How to choose factors (input variables)?

To answer this question, one could probably write a separate book or - at least - a bulky chapter in a book on DoE. In my opinion, it is worth to pay attention to:

Team experience (one needs to create a small DoE team, unless the DoE is a part of a Six Sigma project then the team already exists).
Historical data, e.g. on breakdowns and corrective measures taken.
Multivary analysis.
Knowledge of the "physics" of the process, i.e. the welding process depends on ..., the riveting process depends on ..., the screwing process depends on .... I put the word "physics" in quotation marks because it could also be a chemical or a biological process. It is important to understand its theoretical mechanisms.
Intuition.

I understand that point "Intuition" is very controversial, because what does it have to do with a scientific approach? I will not argue with anyone. However, if someone would like to understand me, let him/her see the history of human discoveries. How many things were discovered, even though experience and knowledge at the time did not indicate one factor or another. In my case, my knowledge of filling machines was negligible, technicians (i.e. team members) indicated several factors and I chose another factor, which was not indicated by anyone, later called "flow meter", although technicians strongly advised against. The more they advised against it, the more I wanted to check this factor. In their opinion, the "flow meters" were perfectly calibrated in an external laboratory. Checking the "flow meters" was a waste of time, in their opinion.

Steps of DoE

Factors and plan selection

DoE makes it possible to study the influence of many factors (acting alone or in interactions) on the output factor, however, apart from the theoretical possibilities of the method, e.g. examining 8 factors, one also has to consider the cost and time factor. I decided to conduct the study with 3 factors. Then I chose the simplest plan: a full-factor 2-level plan that gaves 2 ^ 3 = 8 combinations of factors.

The technicians selected:

Nozzle technical condition (level -1 – worn, deformed nozzle, level +1 – new, straight nozzle).
Technical condition of the diaphragm (level -1 - worn diaphragm, level +1 - new diaphragm).

I insisted on:

Flow meter (level -1 - from the C3-C4 filling machine, level +1 - from the S7-S8 filling machines).

Minitab generates an experiment plan. On the basis of the plan, one can see the sequence of factors change. You had to reckon with 8 combinations of factor settings. Due to the nature of the factors (discrete) and large time-consuming exchange of the flow meter, I decided to follow the sequence shown in Tab. 2, where, as you can see, I gave up randomizing of the combinations sequence. In most of the cases, the sequence of combinations should be randomized, especially for variable factors.

Time consumption of the experiment

It took less than 1 minute to replace the nozzles. Replacing the membrane takes about 5 minutes, while replacing the flow meter takes about 2 hours. Additionally, to reduce the number of flowmeter changes, the experiment was carried out on a C3-C4 filling machine, which meant that the previously installed flowmeter was used in the first 4 combinations. The next 4 combinations were made after installing the flow meter from the S7-S8 filling machine.

For each combination I did 30 fillings for each nozzle separately (left and right), besides, each filling had to be checked on a scale. This gives 30 x 2 x 8 = 480 fillings. The entire experiment needed time of a full production shift and was carried out in non-working time.

Analysis of DoE

This part is simpler than you might think. I use Minitab to create an experiment plan and for analysis. The entire table contained 480 rows. A fragment of the table with the results is in Tab. 3. The last column shows the results of the Y measurements (mass of filling).

In Minitab we choose "Stat> DOE> Factorial> Analyze Factorial Design" and we are only 3 clicks from the results presented in Fig. 4, Fig. 5 and Fig. 6. All the tools complement each other. The diagrams facilitate the interpretation of numerical results.

All the results presented in Fig. 4 are nothing more than a summary of the results of testing all possible hypotheses. Let us analyze some hypotheses:

Nozzle (row marked in blue).
- Issue: Is the "Nozzle" input variable significant?
- Null hypothesis (Ho): Input variable "Nozzle" does not affect the mean of the measurements.
- Alternative hypothesis (Ha): The input variable "Nozzle" affects the mean of the measurements.
- We make a decision on the basis of P-Value (marked with a blue frame). We assume that Ho should be rejected as long as P-Value is less than the accepted significance level usually taken as 5% = 0.05 or 10% = 0.1. P-Value = 0.000, which means it is well below 0.05, therefore, we reject Ho and assume that the variable "Nozzle" significantly influences the mean of the measurements.
Flow meter (line in green).
- Issue: is the input variable "Flowmeter" significant?
- On the basis of P-Value = 0.000 we consider that Ho should be rejected, which means that the variable "Flowmeter" has a significant influence on the mean of the measurements.
- The influence of the variable "Flowmeter" is greater than that of the variable "Nozzle" because the absolute value of T-Value (marked in green box) is greater for the variable "Flowmeter".
Interaction of the variables "Nozzle * Flow meter" (line marked in yellow). An interaction is the simultaneous influence of two or more variables. We can have a synergy effect, that is, a strengthening, or, on the contrary, a weakening effect.
- Issue: Is the Interaction of "Nozzle * Flowmeter" significant?
- Based on P-Value = 0.987> 0.05, we consider that there is no reason to reject Ho, which means that this interaction does not significantly affect the mean of the measurements.

The results presented in Fig. 4 can also be seen in the Pareto chart in Fig. 5 (left) and in the Normal Plot Fig. 5 (right). For example, the effect of the variable C - "Flowmeter" is visible as the largest bar on the Pareto chart and is visible as point C, which is the farthest from the straight line on the Normal Plot. Any point close to a line on the Normal Plot indicates that the mean of the results for that variable or interaction does not deviate from the mean of all results. This means that the given variable is irrelevant, it does not affect the model. Conversely, if the distance is large, it means that the variable is significant. This is a potential control variable.

Another tool is the Main Effects Plot and the Interaction Plot in Fig. 6. The greatest effects can be seen for the "Flowmeter", which is consistent with the highest value of T-Value = 10.40 among data in Fig. 4 (effects table). The interactions, on the other hand, are visible in the graph in Fig. 6 (bottom) as intersecting or converging lines. The diagram shows a clear interaction "Flowmeter * Membrane", which is also seen in the effects table in Fig. 4, where T-Value = -4.08.

Another presentation of the DoE results is presented in Fig. 7. For each combination of factors, there were 60 data for which the short-term process capability coefficient was determined. The best results were obtained for the C3-C4 flowmeter.

A very important note - in the case of this experiment, optimization in terms of reducing the volume standard deviation was more important, as the average of the results is easily controlled by the flowmeter settings. As it is known, Cp is inversely proportional to the standard deviation. In addition to the analyzes presented above, I should also present analogous analyzes for the standard deviation. Unfortunately, I no longer have access to the source data. In summary, the variable "Flowmeter" has the greatest impact on the standard deviation.

DoE Summary

The key to the success of DoE is the appropriate selection of variables, avoiding measurement errors, choosing the right plan, etc. In the early stages it is very important to support team members etc.

However, the last word always belongs to the project leader. In my case, it paid off to be adamant about the choice of factors, but I wish everyone that the factors indicated by the project team were the most important (it is easier this way).

After detecting the defectiveness of the flow meters, we replaced the ones that required it, we have introduced procedures to check their quality with SPC, we suggested to the producer a change in the software and after 2 years I started working on MES on filling machines, which I called MIS - Maco Information System, which works to this day (as of September 2021).

Something for the persistent

I encourage you to test your knowledge in the test on DoE topic. You will find 50 questions there. Only the best have a chance for a full set of points. Emotions guaranteed and satisfaction is priceless ;-).

Please use the form if you have questions about DoE.

Author: Adam Cetera (LeanSigma.pl)
Creation date: 2021-09-15
Modification date: 2021-10-03

Leave a comment below if you would like to add something?