 |
 |
 |
 |
 |
|
 |
Technical ReportWhat’s in this Report? This report describes how the Next DanceTM Retirement Exploration Survey (RES) was conceived, developed, refined and tested for reliability and validity. It outlines the subjective and objective steps that were taken to create the RES and the document that presents the RES results, i.e., Your Next Dance Review. This report was designed to be straightforward and self-explanatory enough to meet the needs of both the non-mathematically minded, as well as the statistically astute. Each step is described in common parlance, as well as in exact statistical language that details the procedures that were performed to attain empirical rigor. It shows how a long list of Dimensions, e.g., aspects and areas relevant to retirement, was generated from a myriad of publications as well as various theoretical frameworks. This list of Dimensions was reduced to 15 simply by removing Dimensions that were redundant and retaining Dimensions that are unique and distinct aspects of retirement. The final set of fifteen Dimensions were:
It describes how for each Dimension, e.g., Financial Security or Attitude Toward Aging, a pool of 10 to 12 items was generated to capture the theme of any given parent Dimension. It shows how the pool of over 180 items was reduced to 60 individual items (e.g.,” I have enough resources to live comfortably”), with each Dimension containing four different but thematically related items. In the RES, a participant rates each of the 60 items on a Likert-type scale, ranging from 1=Strongly Disagree to 6=Strongly Agree. The “score” for a Dimension is the arithmetical average of the ratings of its four items. The RES has evolved over three separate phases of data collection and analysis. Phase I was a pilot study of 57 participants, the results of which guided several modifications to the original RES content and format. Phase II was a second study of 186 subjects that upon analysis afforded further modifications and refinements. Phase III employed 25 subjects and was used to verify the utility of the refinements derived from Phase II. One impact of these three phases on the RES has been to reduce its size, complexity and length of time to complete. Equally important, the reliabilities of the Dimensions were improved and the value of each item to its Dimension was quantified. This Report outlines how a given item was selected for inclusion within a Dimension on a purely mathematical basis. The original Phase I pilot study employed six items per Dimension and these were reduced to the best five items by employing an inter-item reliability statistic called a Cronbach’s alpha. Thus, each and every item was individually statistically tested as to its ability to reliably measure its associated Dimension. The Phase II subsequent analysis, which employed 186 subjects, further reduced the number of items to four items per Dimension, again relying on Cronbach’s alpha to quantitatively identify which items were the weakest and, thus, removed. This Report reveals the “factor structure” of the RES. On a strictly contextual basis, it was anticipated that the 15 Dimensions of retirement would reflect four underlying themes, or subplots. Factor analysis is a statistical method that attempts to uncover these themes by exploring the inter-correlations of all the items with each other. In fact, four factors were found, e.g., Facing the Future. The Dimensions contained in these four factors were reasonably close to the structure predicted. The Report also shows the relationships between an additional question—” My current level of satisfaction with life is…”(1=lousy, 6=great)—and the 15 Dimensions and 60 RES items. Two stepwise, regression equations were computed employing: 1) the Dimensions and 2) the items as predictors of Current Life Satisfaction. The single Dimension that best predicts current satisfaction was Attitude Toward Aging. A second Dimension, Self-Awareness, significantly improved the prediction of current satisfaction. A third Dimension, Commitment to Society increased the correlation even more significantly. The single item that best predicted current satisfaction was, again, an item in the Attitude Toward Aging Dimension. That correlation was significantly improved by an item in the Dimension titled Responsibility for One’s Self. A third item from Commitment to Society further improved the prediction of Current Life Satisfaction. The Report concludes with a strong argument that the Retirement Exploration Survey is certainly reliable and has all the major characteristics of a valid instrument as well. Origin of the Retirement Exploration Survey (RES) Two men and two women with advanced psychology degrees (two PhDs & two MSWs) nearing retirement themselves, with 240+ combined years on the planet and diverse backgrounds in psychotherapy, human resources and business consulting, collaborated to create the list of 15 retirement Dimensions. The original and longer list of potential Dimensions for retirement was gleaned from many disparate sources. These sources included, but were not limited to, books, articles, surveys, Web sites and other retirement exploratory instruments. The concepts and theoretical frameworks of Maslow, Herzberg, McCLelland, Erikson, Levinson, Barron, Rotter, Seligman, Witkin and Attributional Theory in general were also taken into consideration regarding life stage issues, as well as cognitive/attitudinal styles. No single source provided a majority of the final Dimensions. From this list, the authors collaborated to reduce the final number of Dimensions to 15. Independently, the authors generated and pooled item statements that on face value reflected the various Dimensions. For example, the item ”I will be able to retire in the location I live in now” addressed the Residence Status Dimension. There were originally between 10 and 15 items in the pool for any given Dimension. By consensus, the pool of items was reduced to those six that were believed to best represent the different aspects of the same Dimension. To this point, the methodology was entirely subjective and dependent on the clinical experience, collective creative skills, logic, and common sense of the authors. As will be discussed in detail later, the original six items per Dimension were yet again reduced, to five items, and then finally to four items by subjecting each individual item to rigorous mathematical scrutiny (Cronbach’s alpha) as to that item’s relevance and contribution to its associated Dimension. The Retirement Exploration Survey (RES) Format The Survey is divided into three parts. It is a self-administered instrument (online or hardcopy). Part One contains the 60 item statements, 4 items for each of the 15 Dimensions. Part Two has three questions that address one’s current and future life satisfaction, including “My current level of satisfaction with life is __”, where 1 signifies “lousy” and 6 means “great.” Part Three addresses background and demographic information. There are 13 questions pertaining to age, marital status, education, location, employment history, etc. There are two versions of the Survey. One version for “pre-retired” persons and another for those already retired. The two versions are substantially identical with only slight changes in language to reflect pre- or post-retirement status. A respondent is instructed to read each item and rate themselves on a 1 to 6 Likert-type scale, where 1=Strongly Disagree and 6=Strongly Agree. Early versions of the RES asked the respondent to rate each item for different situations or conditions. Examples of those situations/conditions explored were: Current Situation, Desired in Future, Importance, Ability to Close the Gap (between Current Situation and Desired in Future), Expect in Future, and Level of Concern. Early test respondents thought multiple ratings were: too confusing, too long, too many types of ratings, cumbersome, not clear, not uniformly applicable to pre- vs. post-retired individuals, etc. Based on this feedback, in a subsequent testing of the RES, ratings were reduced to: 1) Current Situation, 2) Expected in Retirement, and 3) Desired in Retirement. Then each item had three columns represented by boxes in which a respondent entered their 1 to 6 rating for that item. As discussed later, a pilot study of 57 participants was conducted. Respondents again felt that the Expected in Retirement vs. the Desired in Retirement circumstances were problematic. The averages and standard deviations for these circumstances were highly similar as well, suggesting that they really were not measuring substantially different conditions. Therefore, the current RES was reduced to only two rating scales. Below each item a respondent selects a 1, for “Strongly Disagree” through 6, for “Strongly Agree” for their Current Situation. An identical scale allows a respondent to select 1 through 6 for their “Needed in the Future” rating. All validation procedures and statistical analyses were performed on the Current Situation rating only. Comparisons between Current Situation ratings and Needed in the Future ratings were computed for exploratory purposes but not employed in the validation process. Areas of concern were addressed in a section separate from the 60 items. That section listed the 15 Dimensions by name and asked the respondent to rate that Dimension as of High, Medium or Low concern. A small pilot study later showed that the Level of Concern ratings were not fruitful at all. The rankings ranged from no areas of concern to nearly every area being rated as High. Most importantly, the Level of Concern rankings had virtually no relationship to the individual corresponding Dimensions. Since the reliability of the Dimensions had been soundly verified, the Level of Concern ratings were believed to be the weaker of the two measures. Thus, the Level of Concern section was deleted from the RES entirely. Data Analysis Process The raw data for each respondent was entered into a Microsoft Office XP Excel program spread sheet. Most of the descriptive statistics, e.g., averages, standard deviations, simple correlations, charts, etc., were performed by Excel. The more complex relational statistics, e.g., split-half reliability, factor analysis, Cronbach’s alpha, multiple regression equations, etc., were performed by the Statistical Package for the Social Sciences (SPSS) version 10. Pilot Study Sample and Results To validate the survey, 57 individuals completed it as part of a pilot study. The number of pre-retired and retired persons was very nearly equal. Likewise, the number of men and women were also very nearly equal. The average age of all respondents was slightly greater than 61 years old. Nearly all were Caucasian. The majority of the sample lived in the Northeastern U.S. The average rating for all, originally, 75 items was 4.4 for all respondents, which falls about midway between “Somewhat Agree” and “Agree.” The average standard deviation for all respondents was 0.9, which for a six-point Likert-type scale is very close to the commonly found deviation of 1.0. The Likert-type scale contained “equally appearing intervals” and no true zero. Furthermore, a review of the histograms for the original 75 items revealed that they all had distributions that were quite close to approximating a normal (bell-shaped) curve. Therefore, the two fundamental requirements to perform parametric statistics were met and the analyses could legitimately proceed. The statistical analyses began by computing mathematical averages and standard deviations for all 20 of the original demographic data items and for all 75 of the item-statement Current Situation ratings. Comparing the male and female respondents, there were minor differences between the genders on the demographic data. The women were slightly younger but not significantly so. There were also minor differences between the genders on the 15 Dimensions and a few of the 75 items, but nothing surprising or detrimental to the validation process. Comparing the retired versus pre-retired individuals, there were a few readily predictable differences on the demographic variables. The retired persons were: older, had more children, did more volunteer work, and owned an apartment or condominium versus a home. The pre-retired were: higher educated, more “professional” in occupation, and had greater home ownership. While these demographic variables were statistically different, many were predictable and were not considered to constitute a meaningful disparity. Regarding the 15 Dimensions, the retired respondents were less concerned about finances and felt that work was less central to their sense of identity than the pre-retired individuals. In each case, three of the five items associated with the Financial Security and Work Identity Dimensions were significantly different and resulted in an overall significant difference for the Dimension. There were no significant differences between the retired and pre-retired respondents on any of the remaining 13 Dimensions. Except for the Financial Security and Work Identity items, only four other items were significantly different, but not enough so to create a significant difference for the overall Dimension. Having found no significant differences between the men and the women on the Dimensions and rather trivial differences between the pre-retired and retired respondents, it was justifiable to treat all 57 people as a single sample and to proceed with the validation process on that basis. A larger Phase II sample of 186 surveys was collected to bolster the stability and reliability of all the statistical findings. The results of the Phase II sample were very similar to the means and standard deviations of the pilot sample. The same demographics and Dimensions were different, and in the same direction. Reliability and Validation As mentioned, there were originally 15 Dimensions and six items for each Dimension, whose content seemed logically related to the associated Dimension. Again, a Dimension is assumed to be an area of concern for those approaching retirement or just important to their future. An item attempts to tap into that specific area/Dimension. To objectively evaluate the degree to which the items accurately measured their Dimension, Cronbach’s alpha was computed for each item. “Cronbach’s alpha measures how well a set of items (or variables) measures a single unidimensional latent construct. If the inter-item correlations are high, then there is evidence that the items are measuring the same underlying construct. Cronbach’s alpha is a coefficient of reliability or consistency.” (www.SPSSFAQ.com) Cronbach’s alpha ranges from zero (no inter-relatedness at all) to 1.0 (a perfect relatedness). To determine which items would be retained and which ones would be deleted, Cronbach’s alphas were computed for each individual item for each Dimension to assess the degree of inter-relatedness if that item was removed from the set of six. Usually, to improve something, we think in terms of adding something. The Cronbach’s alpha process operates backward in that something is improved by removing the weak link. For all 75 items, the Cronbach’s alphas ranged from .60 to .91 with an average alpha of .82—which is quite large, suggesting not only good reliability, but speaking to validity as well. For any given Dimension, the alphas were highly consistent with very little difference between them. Generally, the alphas differed only at the hundredth-decimal place, suggesting that they were in fact measuring the same construct. The alphas were so similar that the items that were to be retained or deleted could have been an arbitrary decision. Nonetheless, that item, which when removed from the other five yielded the highest Cronbach’s alpha, was selected for deletion. Thus, the final set of 75 items (from the original 90), five for each of the fifteen Dimensions, were selected entirely on their mathematical relationship with their associated Dimensions. For two of the Dimensions, two items were mathematically tied for exclusion. They had identical Cronbach’s alpha results. In only those two incidences were the content of the items even considered. All other item decisions were strictly empirical. With the larger Phase II sample of 186 surveys, the number of items per Dimension was yet again reduced from five to four via the Cronbach’s alpha statistic, resulting in the current 60 items. The alphas were again quite high with a low coefficient of .69, a high of .95 and an average of .86. In deleting one of the items from the pool of five, typically the difference between the alphas was quite small, generally also at the hundredth-decimal place. Only one item in the Attitude Toward Aging Dimension had a notably deleterious effect on overall reliability for this sample. Of course, that item was removed. The RES was quantitatively shown, therefore, to be a highly reliable instrument with large and consistent inter-relatedness among its items and Dimensions. Reliability can be quantified statistically. Validity cannot be quantified so precisely. The content of the items, however, certainly suggests that they are reasonable examples of an individual abstract concept. For example, the items for the Financial Security Dimension contain words like: resources, funds, no need to work, insurance and financial advisor. The thematic construct certainly appears to be about finances. More importantly and convincingly, however, is the large degree of inter-relatedness of the items to each other. Again, the average Cronbach’s alpha was .86. An alpha for 186 subjects and four variables is significant at the p<.01 level if it exceeds 0.25. An F test is another statistic used to evaluate the significance of alphas. The F-tests associated with each of the Cronbach’s alphas were significant beyond the p<.0000 level. SPSS did not even compute the actual probability, but we know that the odds of obtaining these alphas by chance were something less than one in 10,000. If the Cronbach’s alphas were very low or insignificant, then it may be that the Dimension they address is multidimensional and contains more than one construct. This was far from the case here. At the very least, it can be said that there is far more evidence to suggest the RES is reliable and valid than there is to argue otherwise. Factor Structure Factor Analysis is a method that endeavors to extract fundamental themes, commonalities or subsets from a larger set of variables. On a strictly contextual basis, it was predicted that the factor analysis of the 15 Dimensions within the RES would probably reflect four major themes that encompassed two or more of the separate Dimensions. The found themes are called factors and a specific variable’s correlation to the factor determines to what extent it lies within that factor. A variable’s correlation to a factor is called a “factor loading.” Factor loadings of .7 or greater are considered meaningfully large because then nearly 50% of that variable is contained in that factor. To explore the factor structure of the RES, a principal-components factor analysis, rotated to the varimax solution, was performed by SPSS on the 15 Dimensions. The factor analysis for the pilot sample of 57 yielded a three-factor structure that was interpreted fairly easily. Factor I contained the more enduring traits and attributes. Factor II was about relationships and Factor III captured circumstances such as finances, residence and work identity A subsequent factor analysis for the larger Phase II sample yielded the expected four-factor structure for the pre-retirees. They were: Factor I: Dimensions reflecting core issues related to an effective retirement transition, such as work identity and attitude toward aging. The four Dimensions in Factor I were:
Factor II: Dimensions reflecting concerns about concrete issues, such as finances and residence issues. The three Dimensions in Factor II were:
Factor III: Dimensions related to relationships with others and the community at large. The four Dimensions in Factor III were:
Factor IV: Dimensions reflecting the ability to cope and adapt. The four Dimensions in Factor IV were:
Factor names or labels are arbitrary but attempt to provide an abstraction or concept that captures the content of its variables. To that end, Factor I was labeled “Facing the Future.” Factor II was called “Assessing the Basics” and Factor III was named “Making Connections.” Factor IV was labeled “Coping with Life”. The Dimensions found within these four factors were fairly similar to what was predicted. It is virtually impossible to accurately foretell factor analysis results. A factor analysis does not “prove” anything but rather shows the mathematical inter-relatedness of multiple variables. Regression Analyses After the listing of the 60 Dimension-related items in the RES, a few additional questions regarding life satisfaction were posed. One of those questions was, ”My current level of satisfaction with life is…” (1=lousy, 6=great). The objective of the regression analysis was to explore which, if any, of the retirement Dimensions and their items correlated with life satisfaction. A regression equation is a statistical method that mathematically combines any number of independent variables (predictors) to yield the maximal correlation to a single dependent variable (the criterion). One type of regression equation is built by first selecting the independent variable that has the largest correlation to the dependent variable. In the second step, another independent variable is mathematically selected such that when the second independent variable is combined with the first one, the inter-correlation between the combined independent variables and the dependent variable increases significantly. Other steps are possible as long as additional independent variables added to the equation continue to increase the overall correlation significantly. A stepwise, multiple, linear regression equation was constructed employing the 15 Dimensions as potential predictors of the Life Satisfaction criterion. The first Dimension selected to predict Life Satisfaction was Attitude Toward Aging, with a simple correlation of A second stepwise, multiple, linear regression equation was built employing the 60 items as potential predictors of Life Satisfaction. The first item selected was from the Attitude Toward Aging Dimension, with a simple correlation of r=0.53. The second item selected for inclusion in the equation was from the Responsible for One’s Self Dimension, which raised the multiple correlation to R=0.59. Lastly, an item from The Spouse/Partner Relationship Dimension was selected and increased the correlation to 0.62. These three selected items accounted for about one third of the variation in Life Satisfaction RES Scoring and Feedback in Your Next Dance Review First, the averages for all 15 Dimensions are computed from the ratings of the 60 Current Situation items. Next, the averages for all 15 Dimensions are computed from the ratings of the 60 Needed in the Future items. Next, the 15 Current Situation Dimensions are subtracted from the 15 Needed in the Future Dimensions to yield 15 “Gap” scores. The larger the Gap, the greater the discrepancy between now and what is Needed in the Future. The Current Situation Dimension scores, the Needed in the Future Dimension scores and their 15 associated Gaps are portrayed on the line graph in Your Next Dance Review. Typically, the Needed in the Future scores are larger (higher) on the graph. The average Current Situationscores are generally smaller (lower) than the corresponding Needed in the Future scores. The Gap score is depicted by the shaded area between the two Dimension ratings. The line graph also groups the 15 Dimensions into 4 four clusters. Mathematically it was found that the Dimensions within each of the 4 four clusters have something in common. The cluster titles, e.g., “Facing the Future,” provide a label for the common underlying themes. The line graph provides a quick overview of where someone stands with regard to what he/she feels is Needed in the Future versus his/her Current Situation. It also provides an overall comparison of each Dimension against all other Dimensions. It both foreshadows and summarizes the results in Your Next Dance Review. To provide meaningful results tailored for each person taking the survey, it is essential that each dimension be assigned a score that indicates whether the Dimension is worthy of particular attention, i.e., of high concern, or worthy of note, i.e., of mild concern, or of little or no concern. This feedback is presented in Your Next Dance Review by a traffic-light symbol, with the appropriate color highlighted: • Red Light: The specific Dimension feedback provided lets the respondent know that his/her score is particularly worthy of note as it reflects a high level of concern. The descriptors provided reflect how people with like scores on the Dimension often feel, think or behave. • Yellow Light: The specific Dimension feedback provided lets the respondent know that his/her score is worth a look as it reflects some concern or nagging doubts. The descriptors provided reflect how people with like scores on the Dimension often feel, think or behave. • Green Light: The specific Dimension feedback provided lets the respondent know that his/her score reflects a strength, i.e., a convergence between his/her current situation and desired future state. The descriptors provided reflect how people with like scores on the Dimension often feel, think or behave. Current Situation Score: Red, Yellow and Green Lights In order to provide a tailored, personalized analysis of an individual’s Current Situation Dimension ratings, each Dimension was categorized as being high, low or in the middle of that person’s ratings. To that end, an arithmetical average of all 15 Current Situation Dimensions was computed. If any Dimension had an average rating of less than 3.5, it was automatically designated as a red light. On the opposite end, if any Dimension average was equal to or greater than 5.0, it was automatically designated a green light. All scores greater than 3.5 and less than 5.0 were designated as middle scores, or yellow lights. These criteria were guided by the means and standard deviations of the original samples as well as feedback from early participants. Theoretically, then, a participants could receive all green, all red, all yellow lights or any combination thereof. Gap Score: The Calculation and Significance of Gaps The difference between the Needed in the Future and Current Situation scores on any given Dimension is the “Gap” score. It reflects the discrepancy between where someone is now and where they want to be on each Dimension. A small Gap is generally not of concern. A large Gap may portend a problem or, depending on the individual, may not be an issue at all. There are 15 Dimensions, each with its own associated Gap. Any Gap greater than an absolute value of 2.0, was considered significant. Additionally, any Gap was considered “significant” if it exceeded one standard deviation of all the Gaps. If a Gap was one standard deviation larger than the average of all the other Gaps, then a warning was placed in that individual’s Review for that Dimension. The warning is: “The difference between your Needed in the Future score and your Current Situation score is greater on this Dimension than most of your other Dimensions.” A significant Gap between these scores frequently suggests an area of potential difficulty, but not always. For example, if all your scores are fairly high, it is possible to receive a green light, with very positive statements as feedback, and a warning about the Gap at the same time. Whenever you receive this heads-up about a Gap, it is important to really dig into your results to understand if the Dimension under consideration is one that needs particular attention. It may be the case that the Dimension warrants effort in order to narrow the discrepancy between where you are and where you want to be.” Gap scores do not influence a Dimension’s rating as high, medium or low (green/yellow/red lights). Only the Current Situation scores determine a Dimension’s rating. Interpretive Statements The interpretive text in Your Next Dance Review is written for each Dimension and describes the traits, characteristics, strengths and liabilities common to those people who receive red, yellow or green lights. It is important to remember that a Dimension is assigned a color light based on the individual respondent’s scores only. Respondents are not rated as compared with a national sample or any other normative group. However, for those respondents interested in how they compare with others who have taken their version of the RES, population data is provided at the end of each Your Next Dance Review. In theory, all 15 Dimensions are equally important to a smooth and happy retirement or next phase of life. In fact, all the Dimensions are statistically correlated to an overall satisfaction with a next phase of life. For any given individual, however, some Dimensions will have more importance to them than others. For example, Commitment to Society and Impact of Spirituality vary tremendously from person to person. No attempt was made to weight the Dimensions for a single individual. Each participant is by far the best judge of what is important to him/her and what is less so. As scary as it may be, the respondent is his/her best expert on him/herself. Summary In conclusion, the RES is a self-administered instrument that contains 13 background and demographic questions plus 60 items that are designed to assess 15 separate and important Dimensions related to retirement fulfillment for both retiring and retired persons. The Dimensions and reliability of the instrument are well thought-out and supported by rigorous research. This Report addressed most of the technical issues regarding the creation, design, composition, and statistical analyses and scoring of the RES by Next Dance. If there are questions of a technical nature not covered by this report, please contact us at info@mynextdance.com. |
