Program Assessment

To assess the extent to which CCI students have acquired programmatically-defined knowledge, skills, and abilities and to identify changes that are needed to improve future student learning.

Every program at CCI has learning outcomes that represent that academic degree program’s fundamental units of learning. Here is an example of the Student Learning Outcomes for the Master of Science in Computer Science:

Foundations Core

Outcome (SLO): Upon completing the Foundations Core, MSCS students will have demonstrated the ability to derive properties of algorithms through a theoretical analysis and design algorithms to have strong theoretical guarantees.

Classroom Practice: A strong foundation is the first step towards a successful course of study in the Master’s in Computer Science. In the Foundational Core courses, students will learn data structures and algorithms that will be used throughout their degree program and will carry that knowledge into the job market. Whether students are optimizing system processes or evaluating the appropriateness of various machine learning techniques, these courses go beyond teaching our students the route-based “how” and instead teach our students the higher-level “why” behind real world decision making in the field of computer science.

Career Application: Computational problem solving, algorithm design, and algorithm evaluation.

Skills: Machine learning, Python/C++/Java programming, Statistical hypothesis testing, algorithms (greedy, sorting, classification, etc)

AI, Robotics, and Gaming Core

Outcome (SLO): Upon completing the AIRG Core, MSCS students will have designed and applied algorithms, data structures, and APIs that are core to interactive computing systems.

Classroom Practice: In AI, Robotics, and Gaming, students learn how to program computers to effectively and efficiently interface with humans, reason through statistical models, and interact/react to real or 3D rendered environments.

Career Application: Students will gain insight on A.I., machine learning, and intelligent systems. Graduates might seek functions such as a Data Scientist, Software Designer with A.I. Specialty, or Intelligent Systems Specialist.

Skills: Programming, Machine Learning, Bayesian Networks, Probabilistic reasoning, computer graphics/models/3D rendering

Data Science Core

Outcome (SLO): Upon completing the Data Science Core, MSCS students will have demonstrated an ability to extract insight on a particular problem from a large collection of data.

Classroom Practice: The Data Science Core makes our students invaluable to industry at a time when Big Data is transforming business. Students not only learn the basics of data manipulation, but also excel in communicating ideas around data through the use of engaging visualizations and interactive dashboards. Students are encouraged to explore the intersection of data science and diverse fields such as healthcare, art, business, and education.

Career Application: Students will gain knowledge to prepare them for careers involving database development, data analysis and visualization, statistical modeling and validation, machine learning, dashboard development, data mining, and automated text analysis. Graduates might seek positions in Data Science, Database Management, Database Architecture, or as a Data Engineer.

Skills: Data cleansing, data manipulation, data analysis, R programming/RStudio, big data, data mining

Software Systems Core

Outcome (SLO): Upon completing the Software Systems Core, MSCS students will have leveraged and applied systems level computing, such as parallel computing, networking or computer architecture to solve real-world problems.

Classroom Practice: The Software Systems Core teaches students the brain behind computing systems: networks. By understanding architecture protocols, analyzing performance metrics, and writing executable software-defined networking technology code, students will have the knowledge needed to work in systems large and small.

Career Application: Students will gain knowledge to prepare them for careers involving high performance computing, networking, and computational modeling, parallel and cloud computing. Graduates might seek work as Computer Systems Analysts, Network Engineers, Cloud Computing Specialists, and Application Developers.

Skills: Internet structure, network models, application/transport/network layer protocols, wireless networking, cryptography & network security, programming (Python/C++/Java)

Capstone

Outcome (SLO): Upon graduating, MSCS students will demonstrate graduate level skills to analyze a problem and design, program, and develop a software artifact to address the problem.

Classroom Practice: The capstone experience ensures that MSCS students not only have computing knowledge, but can apply that knowledge through graduate level projects that solve real world problems. As in other courses, capstone projects explore the relationship between computing and other fields. Past students have built software related to event management, electronic learning management, movie recommendation applications, interactive games, and more. In addition to being a culmination of computing skills obtained, capstone projects also require students to work in teams building both their collaboration and leadership skills. Computing skills, problem analysis, application, communication, and leadership are on display in this, the final step of the MSCS journey.

Career Application: The capstone course provides direct experience with software design, development and testing. Depending on their chosen capstone course, graduates can seek positions such as Software Engineer, Software Developer, Software Test Engineer, Game Developer, etc.

Skills: Software process models, agile methodologies (focus on Scrum process), requirements engineering (user stories, acceptance criteria), software design (design principles, software architecture, object-oriented design patterns), software testing (unit and integration testing), and universal modeling language.

How CCI assesses our programs

Every program has a list of Student Learning Outcomes (SLOs) that represent the fundamental units of learning for that program. A curriculum map is created for every program where we identify where the outcomes are introduced, reinforced, and mastered. Data regarding the outcome may be collected at any point in that map, although one would expect progressive improvement as the students move from “introduction” to “mastery.” Each SLO is placed on a three-year cycle to ensure the outcome is studied, improved, and monitored.

Three-Year Cycle

Study (S). Data collection is halted to reexamine the Student Learning Outcome, the assessment process, or educational experience. Two years worth of data will be analyzed and an improvement plan developed and implemented for the following year. In lieu of data collection, in-depth assessment meetings will take place and changes will be documented and approved by program leadership.

Improvement Implementation (I/C). Results of the study year are implemented–new assignments, new assessments, new processes, are rolled out. Bugs should be addressed this year so that the next year, the collection year, is an uninterrupted quiet collection year. Assessment data will be collected and reported on.

Collection (C). This is a quiet period of data collection. It will typically follow an improvement year, so this gives us an opportunity to see evidence of improvement. Assessment data will be collected and reported on.

Collection

Data regarding the SLO is collected in 2 out of the 3 years of the assessment cycle (Improvement Implementation and Collection).

When a course is identified as a collection site, faculty are asked to administer an assessment to measure student achievement of the outcome. The measurement may come in the form of a test, a rubric applied to a paper or project, or even peer assessments. Typically, data collection is completed in the Fall semester, although it is not exclusively completed in the Fall.

Faculty report the findings back to the CCI’s assessment team through spreadsheets or through Canvas’s automated reporting system.

Findings

In January and February, CCI’s assessment team combines data, analyzes trends, and compiles an initial results report.

Faculty and program directors use the initial findings to engage in the feedback process. In Improvement Implementation years, program leadership evaluates the new changes implemented and makes adjustments. In the second collection year, the “quiet collection” year, the stakeholders identify what analyses/research/studies should be launched during the upcoming Study year.

Report

At the conclusion of the year, all work is summarized into a final report that informs the next year’s assessment cycle, and is also collected by the university’s Office of Assessment and Accreditation for accreditation purposes.

Student Learning Outcomes

These five outcomes represent the learning that students will achieve upon graduation from the MS in Computer Science program. The rating levels represent the following: (1) Beginner–not performing at the level expected of a MSCS graduate. (2) Needs Improvement–the student shows movement towards understanding, but the appropriate performance level has not been achieved. (3) Acceptable–the minimum performance level of a graduate. (4) Accomplished–the highest level expected of graduates from this program. (5) Exemplary–beyond expectations of graduates in this program.

1. Upon completing the Foundations Core, MSCS students will have demonstrated the ability to derive properties of algorithms through a theoretical analysis and design algorithms to have strong theoretical guarantees.

2. Upon completing the AIRG Core, MSCS students will have designed and applied algorithms, data structures, and APIs that are core to interactive computing systems.

3. Upon completing the Data Science Core, MSCS students will have demonstrated an ability to extract graduate-level insight on a particular problem from a large collection of data.

4. Upon completing the Software Systems Core, MSCS students will have leveraged and applied systems level computing, such as parallel computing, networking or computer architecture to solve real-world problems.

5. Upon graduating from the MSCS program, MSCS students will have demonstrated graduate level skills to analyze a problem and design, program, and develop a software artifact to address the problem.

Performance 2021

1. Foundations Core SLO–99% passed
2. AIRG Core SLO—100% passed
3. Data Science Core SLO–94% passed
4. Software Systems Core–100% passed
5. Capstone SLO–100% passed

The following graph represents the score distribution. The boxes represent the 1st through 3rd quartile of scores while the line reach out to the max and min scores.

Multi-Year Improvement Plan

Table describing the multi year improvement plan and the SLO cycle of improvement, collection, and study.

Improvement Implementation (I/C). Results of the study year are implemented–new assignments, new assessments, and/or new processes are rolled out. Bugs should be addressed this year so that the next year, the collection year, is an uninterrupted quiet collection year. Assessment data will be collected and reported on.

Collection (C). This is a quiet period of data collection. It will typically follow an improvement year, so this gives us an opportunity to see evidence of improvement. Assessment data will be collected and reported on.

Study (S). Data collection is halted to reexamine the Student Learning Outcome, the assessment process, or educational experience. Two years worth of data will be analyzed and an improvement plan developed and implemented for the following year. In lieu of data collection, in-depth assessment meetings will take place and changes will be documented and approved by program leadership.