In this thesis we explore precision in measurement of buffer overflow and loss probability. We see how buffer overflow probability compares with queuing delay measurements covered in the literature. More specifically, we measure the overflow probability of a packet buffer for various sampling rates to see the effect of sampling rate on the estimation. There are various reasons for measurement in networks; one key context assumed here is Measurement Based Admission Control. We conduct simulation experiments with analytically derived VoIP and bursty traffic parameters, in Matlab, while treating the buffer under consideration as a two-state Markov Chain. We note that estimation error decreases with increase in sampling gap (or in other words precision improves/variance decreases with decrease in sampling rate). We then perform experiments for VoIP and bursty data using NS-2 simulator and record the buffer states generated therein. We see a similar trend of increase in precision with increase in sampling gap. In our simulations, we have mainly considered static traffic passing through the buffer, and we use elastic traffic (TCP) for comparison. We see from our results that the sampling error becomes constant beyond certain asymptotic level. We thus look into asymptotic error in estimation,for the lowest sampling gap, to establish a lower bound on estimation error for buffer loss probability measurement. We use formulae given in recent literature for computing the experimental and theoretic asymptotic variance of the buffer state traces in our scenarios. We find that the theoretical and experimental asymptotic variance of overflow probability match when sampling a trace of buffer states modelled as a two-state Markov Chain in Matlab. We claim that this is a new approach to computing the lower bound on the measurement of buffer overflow probability, when the buffer states are modelled as a Markov process. Using Markov Chain modelling for buffer overflow we further explore the relationship between sampling rate and accuracy. We find that there is no relationship between sampling gap and bias of estimation. Crucially we go on to show that a more realistic simulation of a packet buffer reveals that the distribution of buffer overflow periods is not always such as to allow simple Markov modelling of the buffer states; while the sojourn periods are exponential for the smaller burst periods, the tail of the distribution does not fit to the same exponential fitting. While our work validates the use of a two-state Markov model for a useful approximation modelling the overflow of a buffer, we have established that earlier work which relies on simple Markovian assumptions will thereby underestimate the error in the measured overflow probabilities.

Differential item functioning (DIF) and differential distractor functioning (DDF) are methods used to screen for item bias (Camilli && Shepard, 1994; Penfield, 2008). Using an applied empirical example, this mixed-methods study examined the congruency and relationship of DIF and DDF methods in screening multiple-choice items. Data for Study I were drawn from item responses of 271 female and 236 male low-income children on a preschool science assessment. Item analyses employed a common statistical approach of the Mantel-Haenszel log-odds ratio (MH-LOR) to detect DIF in dichotomously scored items (Holland & Thayer, 1988), and extended the approach to identify DDF (Penfield, 2008). Findings demonstrated that the using MH-LOR to detect DIF and DDF supported the theoretical relationship that the magnitude and form of DIF and are dependent on the DDF effects, and demonstrated the advantages of studying DIF and DDF in multiple-choice items. A total of 4 items with DIF and DDF and 5 items with only DDF were detected. Study II incorporated an item content review, an important but often overlooked and under-published step of DIF and DDF studies (Camilli & Shepard). Interviews with 25 female and 22 male low-income preschool children and an expert review helped to interpret the DIF and DDF results and their comparison, and determined that a content review process of studied items can reveal reasons for potential item bias that are often congruent with the statistical results. Patterns emerged and are discussed in detail. The quantitative and qualitative analyses were conducted in an applied framework of examining the validity of the preschool science assessment scores for evaluating science programs serving low-income children, however, the techniques can be generalized for use with measures across various disciplines of research.

The purpose of the project was to develop an electronic resource that would provide a tutorial for students and offer opportunities for practice of measurement skills.