Has the Mathematical Preparedness of First Year Students at UW
Declined?: A Follow-Up on the Vrscay Report




















                   Susan Coutts
                   (Master's Program in Sociology, UW)


                   John Goyder
                   (Professor, Department of Sociology, UW)



                   17 April, 1998







This project was assisted by a small grant from the Teaching
Resource Office and the Dean of Arts.  We are indebted also to
respondents on the survey, to the Faculty of Mathematics
Associate Dean for Undergraduate Studies together with Gordon
Stubley of the First Year Engineering Office for information on
teachers of first year classes, and to Professor Edward Vrscay
for organizing the interview with informants.



"When to the sessions of sweet silent thought
I summon up remembrance of things past" (Shakespeare, Sonnet 30).


INTRODUCTION

     The quality of education is a continuing preoccupation for
Canadians, and legitimately so for a former resource industries
society that now aspires to compete in the global technology
driven economy.  Canada at the end of the 1950's began to
transform its post-secondary education system from a small
elitist club for the privileged (Porter, 1965) into a broad
meritocracy open to talented applicants of every social
background.  The "Waterloo Plan" that became the centre-piece of
the University of Waterloo (McLaughlin, 1997:36-38) was integral
to the spirit of these aspirations for Canadian higher education. 
It may be inevitable that so ambitious an expansion of education
as that attempted by Canada in the early post war years would
create controversy around issues of the maintenance of academic
standards.  By the 1970's, the Association of Universities and
Colleges of Canada was releasing a report addressing enrollment
and quality issues.  "Whatever the reason," the report stated,
"there is no doubt that large numbers of first year students have
difficulty in writing essays, in expressing themselves
coherently, and in manipulating mathematical symbols and
expressions"  (Barbeau et al., 1977: 20).
     The AUCC report was motivated largely by the ending of
provincial level high school leaving examinations in the 1970s,
but in 1984 a more fundamental critique appeared with The Great
Brain Robbery:  Canada's Universities On The Road To Ruin
(Bercuson et al., 1984).  They pointed precisely to the
accessibility and expansion of universities in the 1960's as the
reasons for declining academic standards.  "Canadian universities
no longer take only the best students and no longer give their
students the best education.  The value, if not the very meaning,
of a university degree has been steadily eroded" (Bercuson et
al., 1984: 7).  
     It was a later book by the same authors (Petrified Forest:
The Crisis of Canadian Universities, in press but excerpted in
the Globe of October 18 1997) that provoked a
counter-interpretation.  Michiel Horn, an historian at York
University, presented an alternative view of Ontario universities
in his October 1997 Globe and Mail article "The Myth Of A Golden
Age In Higher Education."  Horn dismissed much discussion on
higher education as romanticizing of the past, asserting that few
delusions are more widespread than a belief in a lost golden age. 
"Unhappy with what, as we age, we take to be evidence of general
decline, we conjure up a period in the past when life was good,
when children and serving personnel knew their place, when trains
and buses ran on time, and when high quality at a low price was
assured.  The field of education is notoriously prone to such
beliefs. . . ", he contended.  Remembering their own school days
with nostalgia, middle-aged people make a ritual of complaining
about diluted standards.  A look back to Hilda Neatby's (1953) So
Little for the Mind, from nearly a half century ago, gives some
credence to Horn's position.  This early post-war work was
already complaining of "this 'age without standards'" (1953:1)
corrupted by the "progressive education" movement dating back to
the turn of century American philosopher John Dewey.  Clearly,
issues of trends in educational standards must be approached with
an open mind. 
     One October evening such issues spilled onto the floor of
the University of Waterloo Senate.  Dr. Edward Vrscay, a longtime
faculty member in the Department of Applied Mathematics, read
into the Senate minutes a vigourous critique of standards at UW.
The Vrscay Report, as we shall henceforth term it, was re-printed
for the general University community in the UW Gazette of
December 11th, 1996.  
     He took issue with a Senate document intended to outline
future priorities for the University because it "has left out the
one fundamental harm that has been done--the continuing decrease
in academic standards." Although some of the problem was
attributed to problems of reduced budgets and "trying to be
accessible to a greater number of people" at the University
level, a "continued dilution in course material" was in part
traceable to the school system's "decreased academic preparation
of our incoming students."  Even compared to as recently as five
years ago, according to The Vrscay Report, decline in mathematics
skills was evident in the reduced amount of material covered in
courses and the increasing number of students encountering
difficulties with the Applied Mathematics programme. As Vrscay
told Senate:

     Waterloo has been consistently dubbed as a 'leader' among    
     universities.  Goodness knows that we hear enough about this 
     in our own propaganda.  Yes, despite the problems that I     
     have outlined above, Waterloo has been a leader.  However,   
     it could  make a quantum leap and establish itself as an     
     even greater leader if it could overcome the diseases of     
     denial and suppression of the truth and establish itself as  
     a new beacon, not only for postsecondary education, but for  
     the entire educational establishment.  Remember that with    
     all the concerns and doubts about our primary and            
     secondary educational systems, the public will eventually be 
     knocking at our doors.  Indeed, it already has.  There will  
     be no 'epiphany' here, only the need for the truth. 
     
     It was striking in the Senate discussion which followed how
anecdotal trained natural scientists, mathematicians and
engineers became in considering the issues and how sharply
divided faculty were in their perceptions of student quality.  It
was clear to the second author that a social science approach to
Vrscay's issues could be fruitful, and so we set out to evaluate
the question of whether or not the quality of students entering
UW has declined over time.

SPECIFICATION OF THE PROBLEM AND METHODS USED

     We began with a group discussion and interview--one could
almost term it a focus group-- with five UW professors from the
Department of Applied Mathematics, including Edward Vrscay.  This
helped in defining the problem and the most important variables. 
We were directed to first year students as the key group to
examine, since problems of diminished standards or expectations
at the University were linked so emphatically to a perceived
decline in high school standards.  Our sources felt that some
catch-up may occur during the university years, so that the
graduating students are not necessarily egregiously inferior to
their predecessors. It was noted that diluted standards might be
revealed both by the amount of material covered in first year
classes and by the "belling" or upward adjusting of grades. A
problem of administration pressure to bell was cited several
times. One of the informants visualized the problem mainly in
terms of the changing expectations of first year students
regarding the amount of work required of them.  It was argued
that a consequence of the "special me" and "self-esteem" fixation
in the schools was that first year university students
underestimated the amount of work required to acquire
mathematical proficiency. In this argument, once the attitudes
and expectations are turned around, students today perform as
well as ever.  A ten year time frame was suggested for our study. 
The main decrease in the academic preparation of students was
traced to the late 1980's and early 1990's, continuing more
gradually to the present.  To examine a more extended time frame
would also have led to difficulties in finding sufficient
respondents for the opinion survey component of the present
research, due to the recent wave of retirements at the
University. 
     We became aware from the group interview that some
challenging research design issues lay behind the questions
raised by The Vrscay Report.  The main problems are (i) that each
year's incoming class could differ from its predecessor for
reasons beyond the content of the high school curriculum; (ii)
that academic preparation may be subject to lateral "skill
shifts" that are neither higher nor lower than before, simply
different.
     As for the first point, each year's first year class is
shaped by demographic shift, by the priority high school
graduates give to acceptance at UW, and by decisions University
authorities make about the admission standard and size of
incoming class.  In the ideal design, we would have begun the
study long ago, each year gathering data on students' demographic
background, high school grades, IQ, and study habits/attitudes. 
And we would have administered a fixed diagnostic test at the
beginning and at the end of each student's university career.
     Would such a test be even theoretically possible, however? 
The group interview with mathematics faculty raised some doubts. 
It was noted, for example, that with the growing accessibility of
computers and software, mathematical operations that once were
important skills for students to have at their fingertips now are
handled by computer applications.  This could mean a switch from
an operational to a conceptual emphasis that would amount to a
lateral, not a vertical, skill change.  It is analogous to an
occupation that neither gains nor loses skill demands with the
introduction of workplace technology, but instead simply changes
in skill profile.  
     Since The Vrscay Report referred to mathematics, and since
mathematics skills are more easily measured than arts skills such
as rhetoric or essay writing, the study was confined to those
discipines for which mathematical skills are central.  It will be
clear from above discussion why we believed that our research
design should address both "reality"--such as the latter is
measureable given the above remarks--and perception.(1) 
     Thus simultaneously as a search was undertaken for sources
of data for entrance level mathematical ability a perceptions
survey of UW faculty members from certain departments was
designed.  We shall describe first the data from diagnostic
testing in mathematics.

Longitudinal Data on Mathematical Ability

     The Descartes Mathematics Contest was a potential source of
data on mathematical preparedness of high school leavers.  We
were not, however, in a position to separate Descartes
contestants who applied to, but were not accepted, by UW or who
did not finally choose this University from those who did arrive
on campus in September.  Registrar's Office officials told us
that distinction was impossible barring a manual search of
records. Also, the authorities who handle the test were not very
willing to share such results as they did possess with us, and so
we could proceed no further with Descartes data.      
     More useful was the Mathematics Preparedness Test used by
the Faculty of Engineering.  The Mathematics Preparedness Test is
completed by all students in Engineering on the Wednesday of
Frosh Week each September, with results being posted three days
later.  Thus it tests before any exposure to university level
course instruction.  The exam was designed to improve first year
performance and is used as an early identification tool to
determine which students will likely have difficulty in the
engineering programs so that assistance can be sought before
problems arise (Ford, 1995).  The exam was first used in 1990 and
has been conducted each year since.  The tests are said by
informants we interviewed in the Engineering Faculty to be highly
comparable.  Once set, a test is used for three consecutive
years.  The time series covers over 4,000 students tested between
1991 and 1996.

The Opinion and Perception Survey

     Given the impossibility of turning back the clock to test
and interview students from past years, an alternative strategy
was to ascertain the perceptions of instructors of first year
courses at UW.  A survey administered by electronic mail was
designed for this purpose, and cleared through the University
research ethics office.  Initially the Mathematics and
Engineering faculties were chosen for the survey, but later in
the research the faculties of Science and Applied Health Sciences
were added for a confirmatory test of some first
impressions. 
     The survey, consisting of eleven questions, is reproduced in
Appendix A.  A five to ten minute completion time was estimated
for the survey which was initially administered by e-mail on
March 27th, 1997.  Since anonymity could not be provided by a
survey returned through e-mail, respondents were given the option
of printing the survey and returning it through campus mail.  The
sampling for the initial survey began with names of professors in
the Mathematics and Engineering faculties who, according to
listings in the undergraduate calendars for 1986-87 and 1996-97
had instructed at the University of Waterloo for at least ten
years.  Lists of professors who instructed first year courses at
any time during the past five years were then obtained from the
First Year Engineering Office and the Faculty of Mathematics Dean
for Undergraduate Studies, giving the final sampling frame.  The
historical records to capture just those with ten years of
continuous first year teaching were impossible or at least
impracticable to obtain.  The number of faculty who have such
extensive experience teaching first year students is probably
miniscule in any case. We envisioned the survey respondents as
informants who were reporting in part the general impression
within their own faculty, verified by at least some personal
knowledge.
     Since the sample was quite small even with our relatively
loose criteria, the survey was, as noted, later administered to
the Science and Applied Health Sciences faculties to determine if
the results of the initial survey were reproducible.  Similar to
the Mathematics and Engineering faculties, the Science and
Applied Health Sciences faculties were chosen due to the less
subjective nature of grading in those faculties relative to Arts
and Environmental Studies. The sampling frame had to be slightly
re-defined for this Science and Applied Health Sciences sample. 
Although again selecting those with ten years service at UW, as
per calendar listings, we had to use the Course Information  
Report for Fall 1997 to identify recent first year teaching
activity.  The population was thus defined a little more
restrictively for the Science/AHS portion of the survey.  The new
fieldwork took place by e-mail on October 9th, 1997.  Due to a
low response, a follow up copy was sent to the science/AHS sample
through campus mail on Thursday, October 23rd to confirm that all
potential respondents received the questionnaire.  
     Our response rate across all faculties surveyed was still
discouragingly low, about one in three, and so in the winter of
1998 yet another contact was made, by campus mail, with all
nonrespondents.  We were mindful that some one-seventh of a
university faculty contacted in any one term may be hard to reach
due to sabbatical leave, and further suspected that in part the
low 1997 response was attributable to those who, feeling
unqualified to give definitive answers to the issues raised,
self-selected themselves out of the sample.  The new cover letter
stressed that all views would be useful even if the instructor
had not taught first year courses continuously over ten years.  A
question was added to find out exactly how extensive the first
year teaching experience was.  As already reported, the survey
was only sent in the first place to those known from
administrative records to have taught first year students at some
point over the previous five years.  The 1998 questionnaire was
trimmed slightly by deleting some questions which did not prove
important in the first wave. The February 1998 follow-up
generated another 23 cases, and also gave us confirmation that
three people in the sample had retired and were no longer
connected with the University.  
     Table 1 displays the number of surveys returned by each of
the four faculties examined.  

Table 1 - Number Of Returned Surveys By Faculty 
_________________________________________________________________
Faculty                        Returned Surveys   Number Eligible
_________________________________________________________________
Mathematics & Engineering                33                  57

Science & Applied Health Sciences        19                  21

Total                                    52                  78


Response rate=52/78= 67%.
_________________________________________________________________



     The final response rate of 67% is consistent with previous
experience of surveying faculty members at UW and indeed of most
samples surveyed within organizational settings (Goyder, 1987:
ch. 6).  The data base of 52 permits only the most rudimentary
statistical analysis, and yet the survey is far from valueless.
Each of these respondents, after all, has taught hundreds of
first year students.  The depth of experience tapped even from
this limited survey became apparent in the extended write-in
comments volunteered by many respondents, and frequent reference
to this material appears below.

RESULTS  

Mathematics Preparedness Test      

     The Mathematics Preparedness Test administered to first year
engineering students at Waterloo is divided into six different
sections comprising logarithms, geometry, trigonometry, algebra,
analytic geometry and differentiation.   There are 31 questions,
five per section except for six in the algebra section.  The data
for 1990 were incomplete and so we begin the time series with
1991 results.  The 1997 results also are unuseable, because the
test was changed slightly to accommodate its administration to
Faculty of Mathematics first year students in addition to
engineering first year students.(2) Copies of the tests can be
found in Coutts (1997: Appendix B).    
     Raw data summary results from the Mathematics Diagnostic
appear as Table 2 herein.(3)  In every year is seen the normal
distribution-like tendency for most scores to cluster in the
middle range, with tapering tails at each end.  As noted within
the Chi-Square computation at the foot of the table, there is
variation in score distributions year by year at an
extremely high level of statistical significance.  
     A next step in analysis, therefore, was to test for
monotonic association between year and test score, addressed with
a regression analysis summarized in Table 3. 

Table 2: Time Series From the Mathematical Preparedness Test
_________________________________________________________________
Number Correct    1991    1992    1993    1994    1995    1996
_________________________________________________________________
0-1                  0       1       3       7       4       2
2-4                  3      21      31      43      51      38 
5-7                 34      71      60      80     104      70
8-10                74      89      98     131     119      86
11-13              139     122     118     103     127     119
14-16              121     108     110     102     106     125
17-19              116     101      82      70      69      72
20-22              107      59      67      64      63      72
23-25               58      44      47      43      33      32
26-28               29      30      30      19      25      21
29-31               27      14      16       9      10      13

  Likelihood Ratio Chi-Square= 210.867, 50 df., p< .0001
________________________________________________________________
Notes: excludes System Design students. In 1991 there were 32     
       questions.


Table 3: Regression of Score on Year of Test
________________________________________________________________  
  
                   Regression Coefficient    % Variance Explained 
      
                    
_________________________________________________________________ 
    
Year as continuous             -0.180                       1.99
 variable

Terms for individual
 years:
1991 (set to zero)              0.000
1992                           -0.592
1993                           -0.615
--new version of test--
1994                           -1.042
1995                           -1.143
1996                           -0.811                        3.02

Terms for test version & year   
 Version                       -.304*
 Year                          -.102*                        2.10

_________________________________________________________________
All coefficients significant at p< .0001 unless designated
otherwise

*p < .05



     The core result is the regression coefficient
(unstandardized) of -.180 for score regressed on year, meaning
that scores have declined from the early '90s to later in the
decade.  Each category in the results released to us represents
three questions on the test.  If, therefore, the regression
coefficient "says" that scores have declined by .18 points per
year, that means 18% of three questions, or just over one half
question correct per year.  Over the 1991-96 series, then, the
accumulated (and smoothed, by the analysis) average drop amounts
to some 3 questions correct.  The very modest variance explained
of just 2% of the total variation in test scores attributable to
monotonic (uni-directional) decline suggests, however, that this
decline is just a trace in the data, indeed only just detectable
by inspection of the raw figures in Table 2.  
     The year by year coefficients in Table 3 give some leverage
to the issue of a different
test being used between 1991-93 and 1994-96.  Although there was
a relatively large drop in average score 1993 and 1994, when the
change of test was made, in fact the largest single year to year
change occurred between 1991, the baseline year for the time
series (therefore set to zero) and the following year, a drop of
.592 regression coefficient points.  It is  convincing that a
sharp drop in the early '90s, with a tapering later, was
predicted by our informants independently of the time series just
described.  Another assessement of change due to time vs. test
version was effected by regressing score on two predictors; one
representing year and another for version (i.e., 1991-3 vs.
1994-6).  The two terms naturally are closely intercorrelated,
but the significant term for year with version controlled in
effect is reporting that the downward trend is statistically
detectable as a monotonic pattern within each three year cycle of
tests.(4) 
     The downward drift seen in the Engineering Mathematics
Preparedness Test at UW collates with similar  data collected at
the University of Western Ontario by mathematics professor
Christopher Essex (1997). To measure a suspected decline in
academic preparation of first year engineering students at
Western, Essex and several colleagues took a 20-question
mathematics diagnostic test first used in 1984 and
re-administered it to the 1992 class of first year engineering
students.  Both in 1984 and 1992 the test was administered during
the "opening weeks" of university, with the data on the header
page of the test simply changed from 1984 to 1992 for the later
sitting.  Essex (1997) has published results in graphic form in
his own report, but for present purposes we read off the graph in
order to transform his data back into numerical form and
re-analyze them.  In the Western data, we compute the average
decline in score over the nine year interval to be .529 questions
correct per year.  Given that the Waterloo test is out of 31,
compared to 20 questions at Western, the respective yearly
declines as percentages correct are 1.7% and 2.6%. 
     We have no information on how comparable the UW and UWO
tests might be, but are currently endeavouring to secure a copy
of Professor Essex's test.  Access to the test paper will allow a
concern raised earlier to be addressed:  it would be a telling
experiment to present the Essex test dating from 1984 to a panel
of mathematicians who, rather than being asked to take the test,
would be asked to estimate its date. The paper would, of course,
be re- formatted in modern word-processing standards, with all
references to year removed and the panel would simply be advised
that the test dates from sometime between 1984 and the present. 
Suppose such judges consistently could place the test as
reflecting mathematical styles of the mid- '80s (and that of
course could be formally tested for statistical significance
against a null hypothesis of random dating).  Then the
proposition that shifts in mathematical skills have been lateral
rather than vertical (recall earlier discussion) would become
persuasive. 

Professor Opinion Survey      

     Results from the survey of faculty member perceptions of
students collate in an interesting manner with data from the
Mathematics Preparedness Test.  In answer to the fundamental
survey question, 47 percent of respondents felt academic
preparation has decreased against just 12 percent seeing an
increase in preparation today relative to 1987 (see Table 4). 
Forty-one percent perceived no trend, in other words responded
that the academic preparation of first year students has remained
constant relative to first year students in 1987.  The dissensus
among instructors is not surprising in view of how the decline in
scores on the diagnostic test is detectable only with quite
powerful statistical analysis. Both stronger and weaker students
come to the University every year, and the dominant source of
variation derives from these differences between individual
students.  Different professors may
encounter one or another slice of this reality.  As one
engineering instructor told us, "in the last 10 years we have
attracted more of the best possible students than before.  They
are better than before."  But still, the 47% of faculty surveyed
who perceive a decline in high school preparation echo a trend
detected in the Preparedness Test results.
Table 4: From the Survey of Faculty Members
_________________________________________________________________
"In your opinion, has the quality of academic preparation of
first year students in your faculty increased, remained constant
or decreased since 1987?"

% Saying                 Increased  Remained Constant   Decreased

All respondents               12           41                47


"Would you agree or disagree that current first year students
expect to do less work on their courses than their counterparts a
decade ago?"

%                            Agree       Disagree

All respondents*               38           62
                   


"Do you 'bell' (that is adjust) the grades of first year
students?"

%                             Yes           No

Mathematics                    75            25

Engineering, Science & AHS     26            74     
 
                              p of difference < .002
 
"Do you feel pressure to 'bell' the grades of first year
students?"
 
%                            Yes           No

Mathematics                  69             31

Engineering, Science & AHS   20             80

                              p of difference < .001


"Compared to 1987, do you feel the competence of students at the
time of graduation in your faculty has increased, remained
constant or decreased?"

%                        Increased    Remained Constant Decreased

Mathematics                   6                47            47

Engineering                  29                56            15

                              p of difference < .03
_________________________________________________________________
*Percentage includes an imputation to correct for an incorrect
skip  instruction which mistakenly moved ten respondents beyond
this question.  The imputation-- by discriminant analysis using
as predictors perceived pressure to bell, belief that high
schools not sufficiently challenged, and feeling that competence
of UW students by graduate has declined-- adds one "agree" and
eight "disagree's" to the raw count.  In all other questions item 
nonresponse is a minimal one case or so, herein deleted.

On the first two questions, not broken down by faculty, the
difference between Mathematics and other faculties is not .05
statistically significant.



     As already explained, the number of faculty combining ten or
more years service at UW with extensive first year teaching was
just under 80, and the proportion responding to the survey still
less.  Even this tiny data base, however, suggests some
differences in perception across faculty.  The greatest sense of
decline in preparation among incoming students occurred among
mathematics faculty respondents.  These instructors also reported
the most concern about administration pressure to bell grades
upward, they were the most likely to see a problem in student
expectations of reasonable work load, they tended to report
having to dilute the difficulty level of their exams, and they
were the most pessimistic about the quality of UW graduates.  One
informant, on seeing these results, pointed out that even outside
the Mathematics Faculty, in engineering for example, much of the
first year curriculum is devoted to mathematics courses.  Since
most of these are taught by staff from the Faculty of
Mathematics, it is these instructors who are the most likely to
witness, and have to deal with, problems of transition students
encounter between high school and university. 
     On the expectations issue, many write-in comments were made
by survey respondents. One mathematics professor stated "I have
been astonished that my students in Mathematics do not work as
hard as students did 10 years ago.  I can not speak for their
expectations.  They do not live up to mine".  Another mathematics
professor gave this comment: 

     what is a more serious problem is the attitude of students.  
     I think that we have been able to address this issue in our  
     own department undergraduate curriculum.  A clear            
     message is sent to the students that we expect them to work. 
     To their credit, many do respond to the challenge...The      
     modern philosophy of 'self-esteem' and 'outcomes-based'      
     education has been responsible for a decrease in the         
     expectations from students - the material actually being     
     taught is of a more superficial nature (esp. in mathematics  
     and science).  This has naturally led to a change in the     
     attitudes of students.  They have a much different attitude  
     of what 'work' is as compared to students from even 5 - 10   
     years ago.  I think that we have done a good job in keeping  
     the waters of mediocrity from flooding our institution as    
     far as (a) impressing upon students the need to develop      
     critical thinking skills and solid work ethic and (b)        
     teaching them the essentials of their chosen disciplines.  

     According to one engineering professor, students "are
willing to work hard, but they want to be spoon fed in a
different way.  They process information differently".  Another
engineer offered this explanation of the expectations of
students: "I ascribe the changes to a philosophical change in our
treatment of first year students.  Previously standards were set
high and a considerable portion of students flunked out or
withdrew in the first year. This helped motivate all students and
also ensured that the remaining were at least minimally capable. 
Now the philosophy is to retain all entering students if
possible.  The new approach is probably more humane, but can't
help but result in lowered standards."  A mathematics professor
offered a solution to adjust the expectations of first year
students: 

     Today's students are more strongly motivated by fear of not  
     being able to get a good job, and they seem ready to work a  
     little harder...What is certainly clear to me is that,       
     despite the fact that most high school math teachers are     
     competent, students are not given enough work to do.  First  
     year is a terrible shock for many of them.  Also, for        
     what ever reasons, their grades may not reflect their        
     relative  abilities, which is tremendously tough on them,    
     because they are basing their expectations on faulty         
     premises.  The kindest thing we could do would be to         
     institute entrance exams of some sort, just to let the       
     students know where they really stand relative to their      
     classmates.

     Criticism of the school system was widespread across the
survey. One of the science professors specifically expressed
concerns about high school graduates: "I strongly feel that the
public school system shortchanges OAC graduates by failing to
provide them with essential skills in reading, comprehension,
expression (within the bounds of grammar, spelling and
punctuation, that is), and numeracy."  An engineering professor
told us that "the school system in Ontario, as before, does not
challenge the best students".  One mathematician offered an
example of the lack of challenge encountered by students in high
school:  "students will acknowledge that in high school they were
never given a problem they couldn't immediately solve".  Another
mathematics professor expressed fears about changes in high
schools:  "I worry about the changes we are witnessing in our
high schools.  I believe that things that are taught there now
will become part of our curriculum in the relatively near future. 
I think that we are not prepared for this shift of
responsibility".  
     According to another mathematics professor, however, high
school performance is only one of many aspects that affects
students' performance.  This informant said: 

     I have, on several occasions, taught two sections of first   
     year --, giving the same lectures, problem sets, tests, and  
     examinations, and had as much as 7 or 8% variation in their  
     final grades.  Sometimes I can trace it back to incoming     
     high school performance, but more often not.  There are a    
     lot of parameters which affect student  performance.

     About three-quarters of engineering respondents indicated
that they cover the same amount of material in current first year
courses relative to first year courses in 1987.  One of the
mathematics professors, however, thought that 

     students are learning less since [their] curriculum has      
     become 'watered down' (less  courses, less material)         
     ...However, administrators are reacting - quite              
     prematurely and rather irresponsibly to perceived threats of 
     future decreases in enrollments.  There are pressures to     
     dilute the curricula even further. 

     An engineer responded that "students may be less well
prepared, but not less competent".  Similarly, another
engineering professor stated that "students now are as
intelligent as before but are not drilled as well in the
fundamentals of algebra, geometry, English, physics and
chemistry.  Their learning and knowledge seems more superficial". 
 
     A mathematics professor articulated a theme we first heard
in the group interview:

     ...the style of questioning, especially for more complicated 
     problems, has changed.  There has been a trend to structure  
     questions in a way that leads a student from the             
     beginning, along a series of steps, toward the final goal.   
     Rather than just asking,  'prove or show that...', we now    
     first ask the student to write down a relevant definition    
     or theorem, (in essence a pretty big hint) and then take     
     that information, make the  appropriate substitutions, etc., 
     to eventually arrive at the desired goal...problems in       
     current exams...are generally more straightforward and less  
     challenging than in the past.      

CONCLUSION     

     Important gaps exist in our data.  There is no time series
for diagnostic tests for mathematics majors, and the small case
base on the faculty survey is a problem. Nevertheless, some
conclusions do come together from our information taken as a
package. 
     We became persuaded that The Vrscay Report was correct in
pointing to some dilution in the level of mathematical
preparation of high school graduates entering the University of
Waterloo. While the evidence here comes from students entering
engineering, it seems unlikely that trends for mathematics majors
would be greatly different.  Within the context of the general
heterogeneity of students of every year, the longitudinal trend
is subtle.  Indeed it was only due to the statistical power
afforded by some 4,000 students taking the Preparedness Test over
six years that a trend was detectable.  However, even an
estimated erosion of 1.7 percentage points per year cumulates
over a decade into something important.  As noted above, splicing
the data from UW and from UWO together, the trends under
discussion seem to have been under way since the mid-1980s
although they may now have bottomed out.  
     So subtle indeed is the year by year trend that perceptions
among course instructors are highly variant even though on
balance more perceive a slippage in preparation than an
enhancement. The survey data revealed, however, another facet to
the problem.  There seems to be a difference in perceptions among
faculty in Mathematics compared to sister faculties such as
Engineering, Science or Applied Health Sciences.  Such
differences consistently pass .05 statistical significance even
with the tiny case base gathered for the survey.  Judging from
the questionnaires, many mathematics instructors feel
particularly pressured to adjust their grades.  This aspect of
the organizational culture of the mathematics faculty seems to
make frontline instructors particularly sensitive to issues of
standards.  Sociologically, one might almost say that these
different perceptions across faculties are just as important as
the actual documentable decline in high school mathematics
skills.  
     Course instructors who feel pressured to inflate grades can
point to several commentators who see this as a harmful aspect of
the contemporary culture of Canadian university life. Bercuson et
al. (1984), for example, deplored the growing equality between
professors and students and noted classroom course evaluations of
teaching as symbolic of the erosion of faculty authority. It is a
theme echoed by Peter Emberley in a book entitled Zero Tolerance: 
Hot Button Politics In Canada's Universities (1996).  He speaks
of the "empowerment" of students within the Canadian university
system over the past 20 years as a cause of grade inflation. 
According to Emberley (1996: 99-100), "many students believe and
act on the belief that grades are negotiable, others see rules
and regulations as being subject to personal review, still others
think academic decisions are open to endless appeal".  He links
such empowerment with the determination by government to increase
accountability within universities.  Further discussion on the
issue of grade inflation appears in sources such as Easton
(1993), Casas and Meaghan (1995), and Young (1997).
     We can end by underlining the value of diagnostic tests of
the mathematic skills of incoming students.  Given administration
resolve, much more could e done with such data.  The tests might
be linked with background information from the Registrar's files,
or alternatively a short questionnaire gathering background
information might be appended to the test itself.  Given the
sensitivity to downward trend, some independent assessment of the
comparablity of the tests used in different years would seem
advisable.  Such steps would be a convincing form of
accountability and leadership by The University of Waterloo.  For
our part, we are currently exploring the possibility of
constructing further time series by drawing on the exam result
files of colleagues in the Department of Applied Mathematics at
UW.
     
                          FOOTNOTES

     (1) Perception is of course a form of reality that may have
important consequences, but for present purposes these seem the
simplest words for differentiating between the objective and the
subjective aspects of the research problem. 

     (2) Preliminary results show slight improvements in
performance by engineering students relative to past years, but
these improvements have been attributed by the Engineering First
Year Office to the changes in the test not the students.  The
administration of the Mathematics Preparedness Test to the first
year students in the mathematics faculty will prove useful for
future student performance comparisons within that faculty, which
were previously very difficult without a standardized test
written by all students.

     (3) Complete available results by section and overall are
contained in Coutts (1997: Appendix B).  For the most part the
break down of results by section shows few differences in
performance by year.  The trigonometry, analytic geometry and
geometry results show very similar distributions from 1991 to
1996.  In 1995, the logarithms and differentiation results show
slight declines in performance, but these differences are slight
and the results return to similar distributions as previous years
in 1996.  The algebra section has shown the least consistent
results of the six sections from 1991 to 1996.  The performance
of the students on the algebra section in 1991 was very high with
most students answering between four and six out of six questions
correctly.  The 1992 results produce a very different
distribution with many students answering only two or three
questions correctly out of six questions.  The algebra results
improved in 1993, although not to 1991 levels, and remained
fairly consistent from 1993 to 1996.

     (4) Although the test is administered to all engineering
students, making for a population rather than a sample, the
results can be regarded as sample data insofar as we are drawing
implications for University of Waterloo students in general.


                               REFERENCES
  
Barbeau, M., Bruce, I., Clake, J., Morgan, J., Patry, R. &        
 Porter, M., The Role Of The University With Respect To           
 Enrollments And Career Opportunities, Admission Policies,        
 Continuing Education And Community Colleges.  Ottawa:            
 Association of Universities and Colleges of Canada, 1977.  

Bercuson, D., Bothwell, R., & Granatstein, J., The Great Brain    
 Robbery: Canada's Universities On The Road To Ruin.  Toronto:    
 McClelland and Stewart, 1984.  

Casas, F., & Meaghan, D., "Grade Inflation And University         
 Admissions In Ontario: Separating Fact From Perception", The     
 Canadian Journal of Higher Education, Vol. 25, No. 3, 1995, p.   
 49-70.  

Coutts, S. "Quality of Academic Preparation of First Year         
 Students", Honours B.A.  Thesis, University of Waterloo, 1997.

Easton, S. (ed.), Ending The Squeeze On Universities.  Montreal:  
 Institute for Research on Public Policy, 1993.  

Emberley, P., Zero Tolerance:  Hot Button Politics In Canada's    
 Universities. Toronto: Penguin Books, 1996.  

Essex, C., "Education Standards And The Role Of Universities",    
 Organization for Quality Education Forum, Vol. 5, No. 4, March   
 1997, p. 7.  

Ford, J., "Freshman Retention:  An Overall Approach Centered On   
 Mathematics Assessment", American Society for Engineering        
 Education, 1995 Annual Conference Proceedings, Session 1253,     
 California, 1995.  

Goyder, J. The Silent Minority: Nonrespondents on Sample Surveys. 
 Cambridge: Polity Press, 1987.

Horn, M., "The Myth Of A Golden Age In Higher Education", The     
 Globe and Mail, October 28, 1997, p. A23.  

McLaughlin, K., Waterloo: The Unconventional Founding of an       
 Unconventional University. University of Waterloo, 1997.

Neatby, H., So Little for the Mind. Toronto: Clarke, Irwin & Co., 
 1953.

Porter, J,. The Vertical Mosaic. Toronto: University of Toronto   
 Press, 1965.

Vrscay, E., "'Come Back To Reality':  Math Prof Comments On       
 Academic Standards", UW Gazette, December 11, 1996, p. 5-6.  

Young, R., "One Professor's Credo", University Affairs,           
 Association of Universities and Colleges of Canada, February     
 1997, p. 10-11.