Information systems are costly to purchase, deploy, and maintain. Therefore, in a world where business is operated for value maximization according to the theory of rational choice, it is natural to suppose that MIS offers economic value and that this value overcomes the costs. As such it has been an objective of MIS research for at least to determine the economic role of MIS. Today information system effectiveness continues to occupy the highest priority of the MIS research agenda.

Yet much of the research on the effectiveness or success of information systems has relied on psychometric measures of user satisfaction also referred to as 'user information' or UIS. These studies define MIS success narrowly in only subjective terms. In a recent paper Melone [14] argued that the UIS construct is inadequate as a surrogate of effectiveness and that the UIS questionnaire construction and methodology are lacking in scientific rigor. MIS researchers have failed to use the accumulated knowledge and practices of their reference disciplines.

An alternative measure offered by some is system usage which is based on the principle that if the system is being used it must be effective and the more it is used the greater the effectiveness. Srinavasan [15] argues for usage and develops a correlation between usage and satisfaction. More recently Barki and Huff [2] have combined usage and satisfaction scores as dependent variables in an effort to measure success of DSS implementations.

While satisfaction scores and opinions can be useful in comparing two systems within the same user/developer community, they make inter-firm comparisons difficult. These difficulties convinced many researchers to abandon the questionnaire approach and make direct observations of accounting and economic variables that would be indicative of the achievement of business goals.

The problem with statistical power may be minimized if the study is confined to a specific application or implementation or if it is confined to a controlled laboratory experiment. The study of a sales information system by Lucas [12] is an example of the former while the analysis of a simulated strategic information system by King and Rodriguez [10] is exemplary of the latter.

The measurement of IS effectiveness at the firm level as proposed by Crowston and Treacy [5] and Bakos [1] necessarily require that the researcher have a theory of the firm that preludes conflict such as that caused by agency issues [8]. These theories allow for managerial behavior that is not consistent with profit or value maximization. Since IS implementation decisions are made by managers, managerial behavior and motivation are likely to be important variables in the study of IS effectiveness.

It therefore appears that neither a purely subjective paradigm nor a purely functional paradigm of the firm is adequate for understanding the motivations for and the effectiveness of the deployment of information system. There is a wide spectrum within which the concept of effectiveness may be interpreted and the dimensions in which such an interpretation may be placed. Definitional and measurement issues have restored the orderly and scientific accumulation of knowledge in this field.

Three dimensions of effectiveness are identified and explored in this paper. These are the dimension of scope, the dimension of measurement, and the dimension of social paradigm.

The dimension of scope describes how broadly the concept of effectiveness is to be applied. In the narrowest sense it is applied to a single implementation of a specific application program. This can be expanded to include multiple implementations of the same program or to an entire class of applications. We can refer to these as the application level and we may interpret the effectiveness measure in terms of the design, usability, and usefulness of the application in question.

An enhanced sense of generality can be considering the impact of IS on an entire firm regardless of application. At the firm level the effectiveness measure can be related to the firm's MIS organization, policy, budget, as well as attitudes and opinions. Considering an entire class of firms in a homogenous line of business can further increase this generality. An industry or in a broader sense, an entire sector of the economy can be included in what can be termed the industry level. At the economy level the impact of information technology on the entire economy can be assessed as has been done by Jonscher [9] in his famous paper. At the extreme generality of scope, the society, sociologists may consider the impact of information technology on society at large.

The dimension of measurement addresses the type of data to be collected, the method of their collection, and the manner of their interpretation. This dimension can be broadly divided into two parts - direct observations of business variables and psychometric measures of attitude and behavioral variables. Constructing questionnaires designed to assess attitudes and opinions on various types of ordinal scales makes psychometric measures. The psychometric measures can be further sub-divided into two groups; those that measure attitudes (the 'perceptual level') and those that elicit opinions.

The perceptual level lies at the lowest end of the measurement dimension in terms of objectivity. These questionnaires measures the subjective evaluation of users, managers, and IS designers. The user information satisfaction studies can be described by the properties of this level. The influence of subjectivity is lower in the opinion level in which the questions in the instrument are meant to gather data about behavior such as IS utilization and business performance rather than attitude.

The third dimension of IS effectiveness uses the organizational models by Burrell and Morgan [3] and adapted to information systems by Hirschheim [7]. This dimension adds generality to the analysis by allowing for the possibility that not all activities of business enterprises are interpretable as if they were rational organizations objectively seeking to maximize the wealth of the owners. Other paradigms of business organizations exist in which managers may take action to increase their utility rather than the owner's wealth and where various degrees of conflict exist between managers, workers, and stakeholders.

The four paradigms presented by Hirschheim may be placed along a single dimension according to the extent to which they approach the normative model of rational choice. These are, from the lowest level to the highest, Neohumanism, Social Relativism, and Functionalism.

Functionalism is characterized by a high degree of order and objectivity and satisfies the normal assumption of rational choice. In this scenario, the MIS supports rational decision-makers who arrive at optimal decisions for the firm and seek to increase valuation through objective and scientific means. The effectiveness of the MIS can therefore be correctly assessed by performance, however, may not be applicable in other paradigms.

Radical structuralism, for instance, would allow the managers, workers, and owners to each have their own separate agenda. Although each group attempts to maximize utility according to rational choice and objectivity, their goals are allowed to be in conflict.

Social relativism and neohumanism emphasize subjective experiences of individuals rather than the objective goals of groups. These two organizational models differ with respect to the degree of order and conflict. Social relativism combines order and subjectivity. Each individual is viewed as working toward company objectives for the common good but in his own way. In neohumanism, individual goals create conflict. The IS supports the challenges between individuals who are viewed as idealists seeking radical change.

The framework of IS effectiveness presented may be used not only to consolidate past research but also to plan future research. It is hoped that such a structured approach will prove a more constructive way in effectiveness research can build a cumulative tradition and lay a foundation of knowledge. The ultimate aim is to construct a theory of the firm from an IS perspective that can serve as a common point of reference for research in information systems.

References

Bakos, J. Yannis, "Dependent Variables for the Study of Firm and Industry Level Impacts of Information Systems", Proceedings of the Fifth International Conference on Information Systems, 1995 pp. 10-23

Barki, H and S.L. Huff, "Implementing Decision Support Systems: Correlated of User Satisfaction and System Usage", INFOR, Vol. 28 No. 2, May 1990, pp. 89-101

Burrell, G. and G. Morgan, "Sociological Paradigms and Organizational Analysis", Heinemann Press, London, 1979

Crowston, Kevin and Michael E. Treacy, "Assessing the Impact of Information Technology on Enterprise Level Performance", Proceedings of the Sixth International Conference on Information Systems, 1986, pp. 299-310

Hirschheim, Rudy and Heinz K. Klein, "Four Paradigms of Information System Development", Communications of the ACM, Vol. 32 No. 10, October 1989, pp. 1199-1215

Jonscher, C., "Information Resources and Economic Productivity", Information Economics and Policy, 1983, pp. 13-35

King, William R. and Jaime I. Rodriguez, "Evaluating Management Information Systems", MIS Quarterly, September 1987, pp. 43-51

Lucas, Henry C., "Performance and the Use of an Information System", Management Science, Vol. 21 No. 4, April 1997, pp. 908-918

Melone, Nancy Paule, "A Theoretical Assessment of the User Satisfaction Construct in Information Systems Research", Management Science, Vol. 36 No. 1, January 1990, pp. 76-91

Srinavasan, A., "Alternative Measures of System Effectiveness: Associations and Implications", MIS Quarterly, September, 1985, pp. 243-253
 

This page was written and created by a student enrolled in a business course at the University of St. Francis. The content of this page is the work and opinion of the author, not the faculty of staff of the University of St. Francis. Neither the University nor its employees are responsible for the content of this web page.

General questions regarding this web page assignment can be directed to Gerard Kickul.