Measuring+IS+performance

=**Measuring IS performance**= Tuesday, October 13, 2009

Topic overview:
An old truism in business is 'that which gets measured gets done'. As a manager, the task of choosing the right things to measure, finding appropriate ways to measure them, and then interpreting the data properly is an important part of your job. In class today we will look at ways to measure IS performance, commonly referred to as //IS Metrics//. We will cover techniques for measuring the performance of individual information system components, the system as a whole, and the organizations that build, manage, and maintain information systems.

By the end of today's class you should:
 * Be able to explain the difference between measurements of system capacity, performance, and load.
 * Be able to enumerate specific, common metrics in each of these categories (capacity, performance, and load)
 * Understand the connection between selecting appropriate metrics to measure an organization and that organization's performance.
 * Understand the importance of accurate measurement data, and collecting that data in a way that is minimally disruptive

**Preparation for class:** Prior to class you will need to read Chapter 4 of the course textbook [HBP09].

To begin thinking about the ways to apply metrics, write a list of five different ways that you could measure the "performance" of this wiki. Bring your list to class on Tuesday and be prepared to discuss the metrics that you came up with.

**Reference materials:**
Slides: I also suggest reading an older column from Tom Peters on measuring business performance. It is not specific to IT but it makes some very thought provoking suggestions about how to apply measurement to things that are not inherently easy to measure.

Student questions:
As usual, in preparation for today's class, you need to provide at least one thoughtful question raised by your reading of the material you have been asked to prepare for class today. These questions should be posted at least 4 hours prior to the beginning of lecture.

[Bob Monroe] If it is true that 'that which gets measured gets done', applies to an organization, does it also apply to an information system itself? Would an appropriate corollary be 'that which gets measured is likely to perform efficiently'?

[Abdulla Ali Al-Kuwari] The authors claim that "security generates some inefficiency". But, security causes less flaws in the system from outsider, which makes it efficient. So, what do the authors try to say? At the end of the chapter, the author states a warning about the metrics: "do not go crazy". How can a company realizes that it has enough metrics?

[Fatima Hilal] Referring to the IT projects, Peter Drucker, a famous management guru, confirmed that "if you cannot measure it, you cannot manage it". Accordingly, the author stated 5 metrics (web site, SCM, CRM, BPR and ERP metrics) that aid managers to measure and manage their strategic initiatives. He finalized the chapter by saying "Remember to tie metrics to other financial and business objectives" within the firm. How firms can achieve that efficiently and effectively? Is it necessary to implement all the 5 metrics to reach success?

[Muhanna Al - Rumaihi] Figure 4.1 depicts the ranking of e-governments worldwide and their benchmarking efficiency IT metrics and effectiveness IT metrics. The U.S. ranked 1st in efficiency but 3rd in effectiveness. Doesn't having good efficiency IT metrics such as throughput, transaction speed, system availability, information accuracy and response time also cause a positive reaction in effectiveness IT metrics such as usability and customer satisfaction? Are they dependant on each other? If they were first for efficiency why aren't they first in effectiveness?

[Ramsey Massouh] Other than the 5 metrics given the book such as the SCM, CRM metrics ect... are there any other appropriate metrics in which managers can measure IS performance? What if one of the metrics such as SCM fails and so cannot be used to measure the success, how does this affect the overall procedure of measurement? And are effectiveness and efficiency mutually exclusive? I ask this after seeing figure 4.1 where it shows the states is first in efficiency and then in effictiveness is third when talking about e-governments. Does this therefore show that they cant work side by side (effectiveness and efficiency) to make the best product? in this case the e-government.

[Mai Al-Naemi] The author says that efficiency and effectiveness are interrelated and that success in one area does not necessarily imply success in the other. which is more important for an organization: to be successful in efficiency or effectiveness in using a system?

[Shuaa Al-Nasr] The author states that the effectiveness IT metrics are determined by the organization's goals, strategies, and objectives. What other factors (you think) can determine the effectiveness IT metrics ? and how do you think these factors affect the efficiency IT metrics ?

=
[Hira Ahmed] According to the author, some companies do not operate in the optimal area (Figure 4.4) because they may be operating in line with the company's particular strategy. Are those companies better off in efficiency and effectiveness that operate in the optimal area but may not be in line with the organization's strategy like others?=====

[Hadi Murtada] How can companies like google use metrics in order to be more effective and effecient?

[Hatem Salem] According to the author in chapter 4, there is an interrelationship between efficiency and effectiveness. Also, there is a graph that shows that a company can operate and become very efficient but have a minimum effectiveness (top left corner). But doesn’t efficiency lead to effectives or the relationship isn’t relevant? Also when reading about the balance scorecard, I wondered can a company be perfect at all four perspectives or their can exist an opportunity cost when focusing on only one or more of them?

[Hala Khashabi] The balanced scorecard is a management system that “enables organizations to clarify their vision and strategy and translate them into action”, how do organizations ensure that their balanced scorecard system is maintained in the short term and in the long term?  [Yara Saeed] How does a company position itself on the Efficiency-Effectieveness scale? How is the tradeoff made between the two? It seems like if you have a very efficient system but isn't effective in executing your strategy or business goals then it is useless, so in what conditions would an organization opt for such a system.

[Zeyad Al-Mudhaf] Of the 7 metrics (plus or minus two) that a company should choose to evaluate its system's performance, which are the most common across companies that operate in different industries? Are there ones that can be applied to most companies on a broad scale?

[Omar El-Zoheery] If Effectiveness is at a strong pace, and Efficiency isn't, is it still appropriate or necessary to improve efficiency?

[Maryam Al-Kuwari] The author of the book argues that if part of the supply chain management metrics is weak all the other parts are affected. How can we avoid such a thing? and is it possible for the system to overcome the corruption quickly?

[Rifki Bahri] Does a company sees that certain metric has different weight (level of importance) or do they view that all metrics have the same weight? If so, how can they measure which metrics would be more important than the others in terms of measuring their success?

[Laila Murad] Effectiveness and efficiency are interrelated however as the author states having one of them doesn't necessarily imply having the other, which do you think is more important and why ?? does i depend on the industry or one of the tow is always preferred ?

[Omnia Khalid] How does a company allocate importances or weights to the four different sections in the balanced scorecard?

[Laila Murad] Effectiveness and efficiency are interrelated however as the author states having one of them doesn't necessarily imply having the other, which do you think is more important and why ?? does i depend on the industry or one of them is always preferred ?

[Marwan ElGhamrawy] How does metrics really help, i think they are really, just a measure, that cannot be used alone?

[ Aljohara Alobaidly] supply chain maagment matrics is one of the information system matrics that help the managers tomeasure and manage their stratgic initiatives (page 40) how can the supply chain managment help the orgnization ? what areas is covers? and does the good preformance in some parts of the SCM is sufficient ? why or why not?

[Brian Manalastas] Are there consistent sets of metrics that can be replicated throughout any other IS projects or are most tailored to the specifications of the functional requirements?

[Jummana Kahlout] How exactly do business come up with the metrics to measure what initially seems unmeasurable? The book says that companies should stick to 7 plus or minus metrics, but is that per department or overall?

[Benazir Anis]The book states a number of common types of efficiency IT metrics and effectiveness IT metrics.Are there any more metrics that can improve a firm and if so what are they ?

[Eatidal Al-Qatami] How is security important in terms of online shopping -- Amazon for example? How is mainting the best security measures related to the effectiveness/efficiency IT metrics of an online business?

[Noora Al-Mannai] (BPR) and (ERP) metrics considered as types of strategic initiatives. Measuring these types is being extremely difficult. What might be a primary function behind the this difficulty?

[Mathew James] How should a company decide which functions to measure performance on? How often should they benchmark and how should they react if there are many changes to account for and how would this affect the overall efficiency and performance of the company?