Project Check-in

YouTube search... ... Quora search ...Google search ...Google News ...Bing News

• Strategy & Tactics ... Project Management ... Best Practices ... Checklists ... Project Check-in ... Evaluation ... Measures
• Analytics ... Visualization ... Graphical Tools ... Diagrams & Business Analysis ... Requirements ... Loop ... Bayes ... Network Pattern
• Artificial Intelligence (AI) ... Generative AI ... Machine Learning (ML) ... Deep Learning ... Neural Network ... Reinforcement ... Learning Techniques
• Conversational AI ... ChatGPT | OpenAI ... Bing/Copilot | Microsoft ... Gemini | Google ... Claude | Anthropic ... Perplexity ... You ... phind ... Ernie | Baidu

Prompts

• For each Project outcome, what is the Key Performance Parameters (KPP)s measures, leading and lagging, with target dates per fiscal year for next 5 years? 'Leading indicators' are typically input oriented, hard to measure and easy to influence. 'Lagging indicators' are typically “output” oriented, easy to measure but hard to improve or influence.
• What are the Projects (End-State & Retirement) align to the Enterprise Architecture? Not aligned to the Enterprise Architecture?
• What Projects are listed in the Enterprise Architecture?
• What is the vision for our mission environment in Organization's future?
• What mission outcomes does a Component want to achieve?
• What are the gaps in the mission environment to meet our vision?
• For each overarching mission objectives, are we meeting our performance measure targets?
• For each mission outcomes, what progress has been made for prior fiscal year(s)?
• If an mission outcomes didn't meet its target performance measure, why not?
• If an mission outcomes does no longer apply to this fiscal year, why not?
• For each Project objective, what is the Implementation Date for the applicable Measure/End-State?
• What criteria was used for the Component to manage their portfolio?
• For each Project, what is the performance measures, target dates per fiscal year for next 5 years?
• If a Project didn't meet its target performance measure, why not?
• If a Project does no longer apply to this fiscal year, why not?
• What Projects do not support any Organization’s goals?
• What overarching strategic mission objectives does the Organization want to achieve? What are the outcomes?
• What are the number of Components' alignment to each strategic mission objectives? What is the Component side-by-side view?
• What are the number of Projects' alignment to each strategic mission objective? What is the Project side-by-side view?
• For each Project, what percentage (%) objectives are aligned to mission outcomes?
• To what extent are the Key Performance Parameters (KPP)s technically achievable and testable?
• How have Key Performance Parameters (KPP)s metrics performed over the life of the service?
• Which (system) monitoring tools are in place?
• Are there gaps in business capability? If so, are core/priority, Strategic Plan, IT Strategy Plan, etc., linked to identified gaps within the architecture?
• What objective(s) does a Project want to achieve that is currently defined mission objective? Are the Project’s objectives are clear?
• What user needs will this Project (service(s)) address? Why does the user want or need this Project (service(s))? Has the Mission Needs Statement (MNS) been updated accordingly? Which research methods to determine users' requirements were used? What were the key findings? How were the findings documented? Where can future team members access the documentation?

• What are the different ways (both online and offline) that people currently accomplish the task(s) today? Where does this Project fit into the larger way people currently obtain the service being offered? What are the different ways (both online and offline) that people will interact with the Project in the future? What metrics will best indicate how well the service is working for its users?
• What is the scope of the Project? What are the key deliverables?
• Are there architectural view(s) depicting the scope increments (phases) in sync with the Project schedule? Is the linkage clear how capability is being rolled out? Requirement(s) phasing?
• How is the Project’s scope controlled?
• How is this investment intended to improve business operations? How would the Project objective(s) be faceted? [1] Improved efficiencies, [2] reduction of paper-based processes (automation), [3] performance improvement, [4] consolidation/streamlining of redundant activities or technologies, [5] improved quality of data, products, or other deliverable(s), [6] improved customer/citizen experience or service, [7] reduced risk, [8] Increased resilience, [9] cost savings or avoidance.
• What objective(s) does a Project want to achieve that is currently not a defined mission objective at this time?
• What novel (unique) capabilities does the Project provide? Are there similar capabilities in other Components? Is the Project seeking first to use existing systems, services, infrastructure, and/or platforms? (if not using existing systems, services, infrastructure, or platforms, Project must demonstrate a unique requirement and be subject to additional governance and oversight) Do other systems in the Department provide the same or similar functionality? Were these systems considered as a solution? Have these other Projects been leveraged as a resource (lessons learned /experts, etc.)?
• What objective(s) does a Project want to achieve that is currently not a defined mission objective at this time?
• What were the previous year's expected end of year mission results/outcomes for the Project?
• What are the major mission results/outcomes bound by a measurable benefit that are expected to be realized by the Project? How are performance metrics managed?
• What are the milestones of the Project? Please describe the milestones. How frequent are they?
• What were milestone revisions for Project? Why the change?
• What problems/opportunities are being addressed by a Project?
• How is the Project assert to CIO priorities; e.g. network modernization, data center optimization, cyber strategy?
• How is the Project realizing efficiencies through innovative IT solutions?
• Does the Project align to mission areas, goals, critical business priorities, and requirements found in the Enterprise Architecture, directives and external mandates
• Are the requirements being clear and validated? Are the requirements documents validated? If not, what is the status and what specifically is needed to bring to closure? Where applicable are the stakeholders actively involved in the requirements process per appropriate methodology?
• Has the Project performed an analysis human systems integration? To what extent have requirements related to human performance been captured? To what extent have the roles of humans and automation been defined?
• Are efforts being achieved/completed per calendar baseline?
• What are the unique ‘Project Requirements’? e.g. must be disposed not warehoused
• Disposal type determination? e.g. warehousing is allowed, destruction is required. Who makes the determination? What criteria is used to determine type?
• Has the software system design been specified in sufficient detail that a different contractor could continue development without any additional information from the design team?
• What is the targeted average response time for your service? What percent (%) of requests take more than 1 second, 2 seconds, 4 seconds, and 8 seconds? What is the average response time and percentile (%) breakdown (percent of requests taking more than 1s, 2s, 4s, and 8s) for the top 10 transactions? What is the volume of each of your service’s top 10 transactions? What is the percentage (%) of transactions started vs. completed?
• What is your service’s monthly uptime target? What is your service’s monthly uptime percentage, including scheduled maintenance? Excluding scheduled maintenance?
• What are the performance metrics defined in the contract(s) (e.g., response time, system uptime, time period to address priority issues)?
• How does your team receive automated alerts when incidents occur? How does your team respond to incidents? What is your post-mortem process?
• Which tools are in place to measure user behavior?
• What tools or technologies are used for A/B testing?
• Are the installation and configuration parameters of all Commercial Off-The-Shelf (COTS) products identified?
• What mechanisms/procedures are in place to assure mission outcomes are successful?
• If a modernization, list each user community (and the size of the community) that may significantly benefit from this modernization. For each of the user communities listed, what benefits have been identified?
• What is the count of Projects citing each risk/challenge? What are the most cited risks/challenges?
• Why did a life-cycle milestone date change? e.g. termination date delayed?
• What systems of the Project will be modernized? What are the milestones?
• For Project modernized systems, what is anticipated cost savings and cost avoidance?
• For retirement (dispose) Projects, what End-State Project will subsume its 'capability'? What Fiscal year? Have strategies been developed for retiring system(s) that this Project replaces? (I.e., removing h/w, s/w, closing out unneeded interfaces, archiving/subsuming documentation.)
• What Enterprise Architecture activities are migrated on partial Portfolio system migrations?
• When is a partial migration complete?
• What is the detailed description of Full Operational Capability (FOC) for the Project? When did it occur, or planned?
• What Projects receive/require information from a Project at Termination Date?
• If Project does not comply with the Enterprise Roadmap (not aligned with the IT Portfolio), why not?
• Does the Project align to Enterprise Roadmap(s)? Mission? Technical? Is the Project aligned with the IT Portfolio?
• For out years, what does the mission portfolio comprised? When will our End-State environment be realized?
• What are the trends of the mission portfolio e.g. reduction of Projects?
• Who published Projects migration intent and dates?
• Does this investment have a service or services that is or can be used as an enterprise service, a service across more than one component, or a service to an external organization? What Project service(s) is or can be used as an enterprise service, a service across more than one component, or a service to an external organization? Enterprise service reflected in Concept of Operations (CONOPS)? Please describe the service or services and indicate whether it is already in use or can be used.
• What Projects reside in a component's portfolio this year but not last year? Why not?
• What Projects reside in a component's portfolio last year but not this year? Why not?
• What Projects retired (dispose) in previous year?
• For retirement (dispose) Projects, what actions were taken and associated cost avoidance?
• What are the Projects change End-State/Modernization to retirement (dispose)?
• What are the Projects change retirement (dispose) to End-State/Modernization, why?
• What are the Projects are listed as retirement (dispose) (replaced after 3 years) yet remain requesting funding? Why?
• What are the Projects schedules have been 're-phased' or slipped? Why?
• Does the proposed solution architecture align with the Enterprise Architecture (EA) direction and requirements? How does the Project understand what this question encompasses?
• Does Project follow EA Principle 'Enterprise Perspective'? ...to maximize the benefit to the whole of the enterprise?
• Is the Project maximizing the use of shared architecture patterns?
• What supporting function does the Project provide? e.g. ERP, common service, master data, reporting/analytics, tech enabler
• How does the Project map to the Enterprise Architecture Functional Portfolio?
• Is the Project being managed in context of architecture artifacts?
• Does the Project architecture reflect current direction?
• Was the Enterprise Architecture used to identify gaps/shortfalls in capabilities between the current state and the end state?
• Does the Project align to mission areas, goals, critical business priorities, and requirements found in the Enterprise Architecture, directives and external mandates?
• How is the Project managing internal and external dependencies?
• Are the Project's data, reference data, and information exchange models aligned to the Enterprise Architecture?
• Are the Project interfaces (technical, schedule) identified and documented for all dependent systems/Project/organizations?
• Does the proposed Project architecture work across Components to limit duplication? Does the investment currently use or plan to use existing systems/services and/or existing external or industry-provided systems/services? If ‘Yes’, indicate which system/service and describe.
• Does the Project align a Architecture(s)? Which Architectures?
• How is the Chief Architect involved in decisions?
• Do the architectural artifacts fit the Project? Are any missing?
• Do the architectural artifacts reflect Project planning? Are dependencies and evolutionary paths identified? Is there resolution to system level?
• How is the architecture communicated with the stakeholders? How is information about stakeholders maintained? Is the stakeholder’s viewpoint depicted in architectural artifacts?
• Are EA viewpoints used for analysis of Project challenges?
• Who approves architectural artifact changes? Does the PM sign-off? Does a configuration board signoff?
• How is decision rationale pertaining to architectural artifact changes documented?
• Who is responsible for making architectural artifact changes? How is the work partitioned?
• What inputs are provided to the architects? What is the architecture-formation process? The process if the input is unsatisfactory or lacking?
• Is the architecture updated actively maintained and advanced? What is the timing? When were the architectural artifacts’ last updates?
• How is the architecture shared and communicated? Where is it published?
• How are the architectural artifacts used; only to communicate information or as a method to govern the Project? How is Project compliance with the architecture guaranteed?
• Does the architecture communicate best practices to Project team members?
• What is the relationship between architecture and testing?
• What are the unresolved issues pertaining to the architecture?
• What architecture authoring tools are used? What architecture convention(s)/ framework/standard(s) are used?
• What Projects assert to a Business Process Reengineering (BPR) effort? Requesting funds?
• What was the progress against a Project's Business Process Reengineering (BPR) plan?
• For each Project, has Business Process Reengineering (BPR) been completed?
• Have future opportunities been identified to increase efficiency by using technology and Business Process Reengineering (BPR)?
• To what extent has the Project revised the applicable business and administrative processes to improve the opportunity to integrate the elements of the solution?
• To what extent have analysis and/or experimentation and modeling and simulation been conducted with the Project to identify initial user interface, manpower, skills, training, and safety concepts?
• How well do the Functional Requirements trace to Operational Requirements? Functional = What the Project is supposed to do; e.g. detect contraband, verify identification. Operations = How to run the Project's system; e.g. abnormally detection, logging
• Will the Project be impacted by a accounting audit (exclusive of Material Weakness criteria)?
• What Laws, Regulations, and Policies (LRP) provide authority for the Project? Have statutory and/or regulatory authority for the mission(s) is properly cited?
• What Laws, Regulations, and Policies (LRP) impact Project efforts? e.g. accounting principles
• How does the Project learn of new or changed laws, regulations and polices? How are these impacts reviewed and then managed within the Project?
• Has a LRP (Laws, Regulations, Policies) compliance plan been developed?
• Have the complete list of applicable Laws, Regulations and Policies (LRP) in the Enterprise Architecture been mapped?
• What are the Certification requests not recommended by Project?
• Has applicable Personally Identifiable Information (PII) planning and analysis been performed?
• Does the Project collect personal information from the user? Does it collect more information than necessary? How is the user notified of this collection?
• Could the data be used in ways an average user wouldn’t expect? Will any of the personal information stored in the system be shared with other services, people, or partners?
• How does a user access, correct, delete, or remove personal information?
• How can someone from the public report a security issue?
• Does the Project employ Lean Business Agile (LBA) practices? Does Project adopt an iterative and incremental development methodology?
• Does the Project foster continuous improvement of the IT environment with regard to planning, implementing, and maintaining all layers of the EA (business, application, services, data, technology, and security).
• Has the Lean Business Agile (LBA) been used in developing requirements?
• Does the Project have requirements backlog? Is the backlog prioritized periodically based on recent release and stakeholder feedback?
• Is a software tool used to gather, track, plan, and manage requirements backlog? Which version control system is being used? How are bugs and issues tracked and tickets issued? What tool is used? How is the feature backlog managed? What tool is used? How often do you review and reprioritize the feature and bug backlog?
• How does the Project collect user feedback during development? How is that feedback used to improve the Project/service? At each stage of usability testing, which gaps were identified in addressing user needs?
• Has the Lead Business Authorities (LBA) been directly involved in prioritizing the requirements backlog?
• Has the Project established a current state baseline from which to begin Solution Engineering? Are Function Points used? Function Points used correctly? If not using Function Points, what is being used, and how does this compare to Function Points? Are Story Points used?
• How does planned vs. actuals compare using Function Points? Does the story make sense? Is there a sharp rate increase/decrease, why? Is agile working for the Project? (Agile is not always best)
• Has a process been defined to approve and prioritize sprint content & release?
• What is the frequency of deployments of code to production?
• How many days or weeks are in each iteration/sprint?
• How long does it take for a production deployment?
• How long does it take for a new team member to start developing?
• Do Project’s contracts support Lean Business Agile (LBA)? Encourage LBA?
• Does the Project plan to scale Agile development from small development teams to several concurrent development teams (using SAFe, DAD, or other techniques)?
• Do the Project's contracts support Agile methodology implementation?
• Has Project characterization been documented based on the business volumetrics? What is the demand or usage pattern for the Project's service? How many simultaneous users could the Project's system(s) handle, planned/most recent capacity test results?
• Does the approach taken by the Project scale to meet mission needs? To what extent is the planned system capable and/or scalable to accommodate current and projected demands in data volume, users, or transactions? How much capacity is available in the hosting environment?
• What is the scaling strategy for the Project when demand increases suddenly? Has the Project been designed to scale based on demand?
• What is the estimated maximum number of concurrent users who will want to use the system? How many simultaneous users could the system handle, according to the most recent capacity test?
• How does the Project's service perform when usage exceeds the expected target usage volume? Does it degrade gracefully or catastrophically?
• Are Project system-level resource utilization monitored in real time? e.g. response time, latency, throughput, and error rates. Are automated alerts based on this monitoring occur? Are concurrent users tracked in real-time, and monitoring user behaviors in the aggregate to determine how well the Project meets user needs? Are metrics published?
• What is the demand or usage pattern for the Project/service today? What happens to the service when it experiences a surge in traffic or load? How much capacity is available in the hosting environment? How long does it take to provision a new resource, like an application server? How been the service designed to scale based on demand?
• How have Project system-level resource utilization performed over the life of the service?
• What is the level of Disaster Recovery Plan (DRP) that is currently maintained for the system?
• Is a waiver is in place for the Project to accept the risk of a lower level of disaster recovery than required for the system? If ‘Yes’, explain reason for waiver and expiration date.
• What type of agreement is in place for Project's primary service provider to provide disaster recovery services?
• What is the level of readiness of an alternate site for recovery of system? Does it include all capabilities within system boundary, including minor applications and subsystems?
• What is the month and year of the current system disaster recovery plan for the Project?
• What is the scope of the disaster recovery plan regarding all systems within the system boundary for the Project, including minor applications and subsystems?
• What is the maximum amount of time (in hours) to operational status before the Project's system has a failure impact on supported functions?
• What is the frequency of data backup for full backups? For incremental backups?
• What are the types of storage media used for data and source code backups?
• What is the location; city and state, where backup data and system source code is located?
• What are the dependencies on procurement of location or equipment, or existing facility must have proprietary equipment or software to reconstitute the Project's system?
• Where is the planned/current hosting or Cloud Service Provider (CSP) for Project's production systems?
• If Project's system(s) aren't leveraging a cloud architecture, why not? What are the physical, application, or funding constraints for migrating to a cloud environment? Is cloud appropriate?
• If Project systems are planning to leverage a cloud architecture, what is the approach to migrating to a cloud environment? If using a vendor, what is the name of the provider, and what type of service contract will be used? Is the Continuity of Operations (COOP) defining such that it complies with laws, regulations and polices (LRPs)?
• What is the alternate hosting or Cloud Service Provider for where a Project's system(s) would be reconstituted? If the solution is dependent on procuring space of equipment, provide details.
• Where is the alternate hosting location (also include location of Cloud Service Provider (CSP), if applicable); city and state?
• What are the alternate business operations (identify if dependent on teleworking)?
• What are the type of network services in place for accessing alternate recovery site from the alternate business sites?
• What are the network requirements for reconstituting environments?
• What are the services that are outside of Project's system boundary that are required to reconstitute the system? List services and then describe reason for dependency.
• What are the additional mission systems required for operational status for the Project? List systems and then describe reason for dependency.
• What are the challenges or obstacles to optimal recovery for the Project's system(s)?
• What is the amount of annual disaster recovery funding costs; including facility, equipment, backup, and communications?
• To what extent has the Project defined what constitutes a system “failure”?
• What is the amount of funding needed to implement required level of disaster recovery, if needed, that is not in FY 2019 budget for the Project?
• What are the plans for changes or implementation of disaster recovery services?
• Is hazard mitigation, enhanced preparedness, ensuring effective emergency response, and recovery included in needs statement?
• To what extent has disaster recovery been factored into IT strategy? Concept of Operations (CONOPS)?
• Are roles and responsibilities pertaining to contingency operations clearly defined? Have all participants confirmed commitment to contingency planning? Responsibilities in onboarding procedures?
• Does the Project have a Contingency Plan? Does it address known and potential threats?
• What scenarios and use cases have been defined? What threat (e.g., natural disaster, illegal activities, cyber and terror) or business activity did the scenarios describe? How did the scenarios enable defining alternative methods to prevent, intervene, or respond to the threat or business activity?
• Have interdependencies been mapped? What dependencies (other Projects/systems) does the Project have? Are they a risk? Do they support/interface the chosen technology? How does the mapping influence communications?
• How complex are the planned interfaces? Is this risk captured in the Risk Register (RR)?
• To what extent has disaster recovery been factored into testing? Linked to requirements?
• Is there a linkage between managed risks, architecture, and infrastructure?
• What would be the impact of a prolonged Project downtime window? What would be the impact of a catastrophic data loss?
• Has the Project addressed reliability? What are the Project's service(s) monthly uptime target? What is the monthly uptime percentage, including scheduled maintenance? Excluding scheduled maintenance?
• How is the Project paying for hosting infrastructure (e.g., by the minute, hourly, daily, monthly, fixed)?
• Is the Project's services hosted in multiple regions, availability zones, or data centers?
• Did contingency planning leverage the Enterprise Architecture to assure known dependencies were accounted for?
• Are there Service Level Agreements (SLAs) for provisioning internal services? External services? Do the SLAs include remedies; for failure to provide acceptable performance, time frames and escalation procedures. Define a process for monitoring, tracking, evaluating performance and resolving poor performance? Does the SLA include critical timelines, e.g. how quickly DHD has access to its agreed-upon resources, hot site, network recovery, fail-over, recover and restart downed system via a recovery service.
• When was the month and year of the latest system disaster recovery test for each system of the Project? How did actuals time/data meet the defined requirements? If requirement not met, what is the plan?
• What type of disaster recovery test(s) that were conducted? If more than one test was conducted, explain.
• What was the result of the latest disaster recovery test? If failed or passed with conditions, explain.
• Does Project implement SecDevOps/Continuous Integration/Continuous Delivery?
• What percentage (%) of the code base is covered by automated tests?
• How long does it take to build, test, and deploy a typical bug fix? How long does it take to build, test, and deploy a new feature into production?
• Does operations or test results perform in accordance with operational or functional requirements?
• Is the Project’s software tested independently concurrent with development (i.e., no completion of the full software release before handoff to Quality Assurance testers)?
• Have all the operational/functional requirements been reviewed by the acceptance test team to ensure that the requirements are clear, meaningful, and testable? Including Project documentation in addition to the Operational Requirements Document (ORD).
• Are the infrastructure requirements defined for the Project? Have the decomposed system requirements been reviewed by the acceptance test team to ensure the requirements are clear, meaningful, and testable?
• Has user acceptance testing identified any gaps in required capabilities?
• What test tools are used? Which deployment automation or continuous integration tools are used?
• How does the IT Project adhere to the department-wide IT security Project?
• Does the Project have a patching plan/methodology in place?
• Any outstanding security alerts? Are all patches up to date (preventing zero-day attacks) with interface software?
• Does Project follow Enterprise Architecture (EA) Principle 'Security in Depth'?
• How does the Project identify cybersecurity/resilience concerns? Is cybersecurity adequately addressed/planed into the Project?
• Does the Project have an appropriate System Security Plan (SSP)?
• Does the Project have an appropriate Security Risk Assessment (SRA)?
• Have cybersecurity/resilience concerns design into the Project from the beginning?
• What is the threat landscape for the Project and planned threat assessment activities? Is threat modeling/management prepared and achieved at the Project or elsewhere? What approach is the Project taking to manage threats?
• Does the Project address cybersecurity risks in manner identified via process and policy? What is the Project status with cybersecurity testing? Planning?
• Are Project cybersecurity risks continually closed out or mitigated?
• Are Project's cybersecurity risks captured within the Component's Risk Assessment Report (RAR)?
• Are appropriate cybersecurity stakeholders involved in identifying cybersecurity risks? Is identification rationale for recognition sound?
• What is Risk Management Framework (RMF) implementation strategy and associated activities?
• Has the cybersecurity risk governing body provided the overall cybersecurity risk recommendation for the Project?
• If edge technology (e.g. sensors, mobile), are these listed? If not, why not? Which edge devices were not considered part of the security testing?
• Was the Enterprise Architecture's security features (also known as controls) addressed in the security authorization process to assure the investment will be granted an authorization to operate?
• Have cybersecurity risks been appropriated identified, evaluated, and addressed, as required for obtaining an Authority To Operate (ATO)?
• Any new hardware added to the configuration after security testing, since last ATO?
• In coordination with the component CISO, has the Project established a process for the continual updates to the Risk Assessment Report (RAR), post- ATO (continuous monitoring)?
• Is ATO on the critical path? What is the status in obtaining the necessary ATOs?
• Does the Project follow data management policies, access procedures, and standards to maximize information security?
• Is there an Information Security Continuous Monitoring (ISCM) waiver?
• Does the Project have an updated Security Requirements Traceability Matrix (SRTM)?
• How does the Project plan to secure its network communication and data?
• If external cloud or cloud hybrid, how does the Project plan to secure its data?
• Does the Project have performed a Cyber Resilience Review (CRR)?
• Have appropriate classified, open source, and operational history sources been consulted when characterizing the threat to the mission?
• Are the threat actors and intent mapped to the appropriate parts of the operational level technical architecture in the Operational Requirements Document (ORD)?
• Are the threat actors and intent mapped to the appropriate parts of the data architecture in the Operational Requirements Document (ORD)?
• Are operational impacts of threats well-defined and decomposed into observable and quantified supporting measures?
• Has the Project performed an analysis of information security needs and does it justify the FIPS 199 information security categorization? What is the current overall security assurance level (SAL) for each of the systems in the Project? What is the ratings for Confidentiality, Integrity, Availability, for each system?
• Does the system design provide the security reports needed to audit and monitor the Project in production?
• If 'standalone' (non-networked) systems how are the patches scheduled (reducing zero day attacks)?
• In respect to security, what is the maturity of each system? What is the next step?
• What cybersecurity assessment tool(s) are being used with the Project?
• What Cybersecurity standards were selected? How compliant is the Project for each standard selected? Who selected which standards need to be addressed?
• Does the Project report on Cybersecurity Project Health Report? If so, what is the Project’s current rating? What is the supporting justification for the rating? ii. For each Project’s systems, what is the overall rating, and what are the Scorecard Metrics ratings for Software Asset Management (SWAM) defects, Vulnerability defects, Configuration Settings Management, Malware? What is the justification for the ratings? Any changes from last review cycle, if so, why?
• Cybersecurity Maturity – What is the Project’s cooperation environment? [1] the Project does not have mechanisms in place to coordinate with partners, [2] the Project understands its role within the ecosystem, but has not formalized its coordination mechanisms, [3] the Project understands its dependencies with partners and has in place risk-based mechanisms to respond to events, or [4] the Project manages risks and actively shares information with partners to improve cybersecurity before a cybersecurity event occurs.
• Cybersecurity Maturity – Addressing the Project’s risk management, what best describes the environment? [1] managing cybersecurity risk has not been established, with cybersecurity risk management implemented on a case-by-case basis, [2] risk management is not across all aspects of the Project, but approved processes are defined and performed on an informal basis, [3] Approved procedures are in place, personnel trained, processes are implemented as intended and reviewed, or [4] Cybersecurity risk management is part of the culture of the Project, where information is shared, processes evolve with a continuous awareness of activities on Project systems and networks. Incident response capabilities are part of the Project’s solution.
• Cybersecurity Maturity – What is the maturity of the Project’s risk management? [1] Not formalized, [2] Approved but may not be established across the Project, [3] Formalized and updated, or [4] Incorporates Risk Management across all aspects of the Project, linked to Component’s overall risk mitigation process, uses advanced cybersecurity technologies and practices through a process of continuous improvement?
• Cybersecurity Maturity – What is the maturity of SecDevOps for the Project? Are embedding continuous verification and detection to stop untrusted or compromised users, devices and workloads from accessing applications and the network used across all systems?
• Cybersecurity Maturity – What cyber security Defense in Depth layer(s) does the Project enable?
• Cybersecurity Maturity – What capabilities (per the cyber maturity model) does the Project enable?
• Are User and Entity Behavior Analytics (UEBA) and machine learning (ML) used to help to create a baseline for trusted workload access?
• Where is the Project in moving toward a zero-trust architecture?
• ... Are all stakeholders represented in the architecture? Data? Network? Instrumentation/sensor devices? Mobile devices, Applications? and workloads affected?
• ... Is how the data is used and accessed (roles) well documented?
• ... Are Laws, Regulations, and Policies (LRPs) mapped to data flows?
• ... Does the Project make use of micro-segmentation; to create secure zones in data centers and cloud deployments that allow you to isolate workloads and protect them individually? Are there zones within the Project using micro-segmentation to address legacy applications?
• ... Does the Project implement software-defined access (SD-Access) to enforce application and network access based on dynamic context; mapping context and devices to scalable groups thus simplifying end-to-end security policy enforcement?
• ... Does the Project/Component have a cybersecurity ontology to support information integration and cyber situational awareness in cybersecurity systems? If so, is the ontology used by different cybersecurity systems for information sharing and exchange? If so, is the ontology used between threat modeling and defense tactics incorporated? What attack taxonomies have been adopted?
• Does the Project need to incorporate managed endpoint detection?
• ... What protection components are included in the Project’s endpoint security solution?
• ... Is there a single management console for all the endpoint security components?
• ... What percentage of devices are covered today? Planned at end-state?

• Is the Project following the Cybersecurity Systems Engineering - Implementation Guide? If not, why not?
• For non-IT (hybrid IT) with embedded IT, how is the Project addressing cyber?
• Has sustainability been adequately planned to include resource needs (staff and cost)?
• How and how often is the Project and its system(s) and service(s) tested for security vulnerabilities? When was the last test time and result for each?
• How does the Project identify cybersecurity/resilience testing processes?
• What kind of cybersecurity threats will/have been tested?
• What methods will/were used to test cybersecurity threats? What are the resulting risk mitigations?
• How will/are testing results and recommendations incorporated into the Project?
• When was the date of the last risk assessment report? Are the results published quarterly?
• ... When was the date of the last version of the Project’s Systems’ security plan(s)? Are the plans published quarterly?
• ... When was the date of the last Threat Assessment and Vulnerability Assessment? Is the assessment published semi-annually?
• Did the Project pass Office of Chief Information Security Officer (OCISO)’s automated tests? If not, what is the status?
• Does the Project design employ vendor-neutral open architecture standards?
• Does the Project utilize standards; architected for interoperability? Which ones are cited/used?
• Does the Project follow Enterprise Architecture data management policies, access procedures, and standards to ensure data remains available in shared environments?
• Does Project have Logical Data Model (LDM)? Have the data supporting the business processes been specified to a conceptual level? Logical Design Document (LDD)?
• Does Project have a Data Management Plan (DMP)? Have data management requirements been defined?
• Does Project have a Data Quality Plan (DQP)?
• Does the Project have an updated Data Insertion Package (DIP)
• Has Project created and filed all appropriate Acquisition documentation that is required based on current lifecycle?
• Does the Project have details about how each system will function at Full Operational Capability (FOC)? System Design Doc (SDD)?
• Does the Project identify an applicable updated Mission Need Statement (MNS)? What specific Doctrine, Organization, Training, Materiel, Leadership and Education, Personnel, Facilities, Regulations, Grants, and Standards (DOTMLPF/R/G/S) recommendations does/will this Project address?
• * Does the Project identify a updated Capability Development Plan (CDP)? In defining the maturity of specific technologies being considered for proposed materiel solutions, to what extent does the Capabilities Development Plan (CDP) support adequate evaluation of mature or feasible technologies?
• Does the Project have an updated Operational Requirements Document (ORD)? Are operational requirements (as described in the ORD) valid, validated, and complete (by Sponsor), testable, and measurable? Do they support the original mission need as stated in the Mission Need Statement (MNS)? Has requirements interdependency been considered and/or analyzed? Has an ORD been developed that captures Initial Operational Capability (IOC)? Minimum Viable Product (MVP)? Full Operational Capability (FOC)?
• Has the Concept of Operations (CONOPS) adequately defined the “to be” business process?
• Does the Project have an updated Concept of Operations (CONOPS)?
• Does the Project have an updated Requirements Traceability Matrix (RTM)?
• Does the Project have an updated Functional Requirements Document (FRD)?
• Does the Project have an updated System Requirements Document (SRD)?
• If Commercial Off-The-Shelf (COTS), what are the dependencies? Are the dependencies of COTS products identified in the architecture?
• Which interfaces are using an proprietary industry standard? Is the version up to date? If not, why not? Is there a standard and is not being used, why not?
• Which interfaces are using an open standard? Is the version up to date? If not, why not? Is there a standard and is not being used, why not?
• What other known Projects use each interface? Has there been exchange of information, code, lessons learned?
• Are Project's interfaces defined well enough to drive solution development? What are the data exchange dependencies for the Project?
• Is there a type of interface for every user/machine with the Project?
• What are current identified Project interfaces, which are proposed/planned?

Internal interfaces: If asked, the same question, would trading partner list this interface as well?

• External interfaces: Which are public facing?
• External interfaces: Are all security patches up to date with interface hardware and/or software?
• Is the interface shown on an architecture artifact? Which artifact(s)?
• What are all of the interfaces, and include all relationships internal and external to the Organization?
• Are all the availability calculations (algorithms) agreed upon and documented in the Service Level Agreements (SLAs)?
• Are there interfaces without Service Level Agreements (SLA), if so, what are they? Are SLA being met, if not why not?
• Are the Project's technologies identified consistent with the target Technical Reference Model (TRM)? What items are approved in the TRM? Component’s TRM? For software not in a TRM, what is the schedule for getting software approved? Any software being used and is on the ‘Not approved’ list, why?
• Is the Project leveraging Technical Reference Model (TRM)-approved software where possible?
• Are modifications to the EA necessary to accommodate the Project and have they been through the Technology Insertion (TI) Decision Request Process? Does the Project have an updated Technology Insertion Package?
• Have all created services been added to the Service Catalog and submitted to the EA PMO for registry in the service component reference model?
• Is the hardware shown on an architecture artifact? Which artifact(s)?
• Is the software shown on an architecture artifact? Which artifact(s)?
• Does the proposed Project system(s) design reflect a modular architecture?
• Does the Project map system functions to capabilities they will support at Full Operational Capability (FOC)?
• Does the Project have an updated “as-is” architecture diagram and business processes defined? Are they reflected in the Operational Requirements Document (ORD)?
• Does the Project have an updated “to-be” (end-state) architecture diagram and business processes defined? Are they reflected in the Operational Requirements Document (ORD)? Are dependencies shown/described?
• Does the Project have the updated “to-be” (end-state) architecture diagram depicts the evolution/phases of the Project, as the Project is to be managed?
• Have business processes been specified to a logical level for the Project? Have the Project processes been documented (e.g., use cases, flow diagrams)? Are all system and functional requirements accounted for in the design?
• Do the Project's priorities listed for each requirement accurately represent the business' capability needs?
• Have the functional requirements been logically decomposed to an acceptable level of detail (at least to the major subsystems or software components) in the Project's system requirements? Have all functions in the logical design been allocated to the system design?
• Have requirements been updated based on the user review of the proof-of-concept, or technology demonstrations? Is the requirements baseline stable and configuration controlled?
• Will the Project provide all the business capability as planned? Have any new business capabilities been identified?
• Does the Project include all items assigned to it for each release?
• Does Project follow EA Principle 'Minimal Technical Complexity'? ..sharing or acquiring services, infrastructures? Is the Project using shared services, data consolidation or for the use of cloud computing in the Investment “to-be” aligned with the Enterprise Architecture?
• What percentage of the Project's capabilities are manual processes? Is there an opportunity to automate?
• How is the Project leveraging new technologies? Have disruptive technologies been considered? Have existing R&D activities been considered when identifying where materiel solutions may address the capability gaps or business need? Describe how the investment is leveraging new technologies.
• Is machine learning being applied to Project? Have processes been reviewed for opportunities?
• What is the number of instances of a Project systems, e.g. changes impact what number of installations? Where are they located?
• Does the Project, where possible, leverages existing systems, services, infrastructure, and/or platforms?
• What 'Hosting and Platform Services' are planned or implemented by the Project?
• What 'Digital Identify Management and Access Control Services' are planned or implemented by the Project?
• Does the Project have system(s) have or have planned mobile applications? If Yes, describe your scope, audience, and objectives.
• To what extent has the Project's capability been assessed for technical feasibility and maturity (i.e., is the technical solution already possible or possible in the near term)?
• What technical risks associated with the Project have been identified relevant to their integration?
• Has an illustration depicting the conceptual network been developed and documented for the Project? Has the technical infrastructure been specified to a conceptual level? Have locations and types of infrastructure components been identified and documented?
• Does the Project include all technology assigned to it for each release?
• What is the Project's development stack? Why was this stack chosen?
• What items will/are made available to the public as open source? If the codebase has not been released under an open source license, explain why. What components are made available to the public as open source?
• What datasets are made available to the public?
• Does the Project have an updated Service Reuse Plan (SRP)?
• What extent have the Critical Technology Elements (CTEs) been identified for each Project's system/solution considered?
• To what extent have the Critical Technology Elements (CTEs) been assessed, both individually and together, to determine their maturity in the specific application for the proposed materiel solution?
• To what extent have the Critical Technology Elements (CTEs) assessments been included as input to the Analysis of Alternatives (AoA)/Alternatives Analysis (AA)? To what extent have evaluation criteria been developed based on applicable technical and mission/business needs to discriminate among technical alternatives?
• Does the system connect to a Network? If so, which one?
• If not and on a network, which one (as well)?
• Do the network meet the technology’s requirements?
• What supporting technologies is the Project planning to leverage? Are there any risk from these selections?
• What are all the associated hardware being used?
• Is the hardware shown on an architecture view? Which view(s)?
• If edge technology (e.g. sensors, mobile), are these listed? If not, why not? Which edge devices were not considered part of the security testing?
• Any new hardware added to the configuration after security testing, since last ATO?
• Any new software added to the configuration after security testing, since last ATO?
• What are all the associated software being used? Which are embedded?
• Where is the current IT hardware/software located?
• Any licensing/migration dates which are considered a risk, which products, and what is the go forward strategy? Are there problems with a vendor? A product to be supported in the near future?
• Does Project match their data artifacts based on Enterprise Architecture alignments?
• Has the Project implemented data stewardship to proactively manage authoritative sources, data interpretation, context, and document in the Enterprise Architecture?
• Has the Project's data architecture been reviewed to assure the design will meet capacity functional and performance requirements?
• Does Project follow EA Principle 'Data as an Asset'? ...emphasizing data quality and sharing? To what extent have concepts for data storage, data tagging, and data sharing been defined?
• Does the Project have system(s) considered a “big data” implementation?
• To what extent will the data need to be modified/reformatted to support the new data structure and will the modification be able to be completed in an automated way? To what extent will legacy data need to migrate into the new system so that the old system can be decommissioned? To what extent will legacy data need to be cleaned up before migration? Does the data conversion plan (documented in the data management plan) account for possible cleansing and data quality issues as well as performance impacts to the existing Data Architecture?
• Does the Project's data retention requirements meet the business need?
• Has the Project itemized data architecture alternatives? Have the alternatives been categorized, prioritized, and cost-justified?
• Is the data required by the Project already available or will it be made available?
• Which database(s) are the Project using? Why were they chosen?
• What datasets will/are made available to the public? Publishing location?
• Have sharing opportunities been identified to reduce redundancy and increase data integrity?
• For Project modernized systems, who are the user communities (and the size of the community) that may significantly benefit from this modernization?
• Who are the Project's stakeholders? What organizations use the Project? Who are your users? What organizations does the Project need to be successful? Who are actively involved in the Project, is affected by the Project's outcome, or can influence the Project's outcome? Is the identification rationale/approach sound?
• Which users will have the most difficulty with the service? Why? Has the difficulty been addressed?
• How often are users feedback solicited? Are the Project's findings about user goals, needs, behaviors, and preferences documented, and shared with Project leadership?
• What are the user's pain points in the current way people accomplish the task? If a user needs help while using the service, how do they go about getting it?
• How does the Project measure customer satisfaction?
• How does the Project/service’s design visually relate to other government services?
• Are prioritized list of tasks the user is trying to accomplish, also known as “user stories” recorded and managed? What metrics will best indicate how well the service is working for its users?
• If consolidation, which Project interface will be browned out? Have stakeholders been notified and agreement as to dependency mitigation planned? When/how it the legacy Project being switched browned out?
• Are approved operational requirements documented where user requirements are documented?
• Was the Enterprise Architecture referenced when identifying impacted stakeholders?
• Has the Project performed/planning a survey to determine if the solution may be used to improve mission capabilities common to components?
• Have users and operators been fully engaged in developing and validating the needs and requirements as reflected in the Mission Need Statement (MNS)? Operational Requirements Document (ORD)? Concept of Operations (CONOPS)?
• Do the reporting requirements ensure that the business users get the information they need?
• What is the Life Cycle Cost Estimate (LCCE) Rough Order of Magnitude (ROM) Cost for Project?
• Is there a clear breakout of funding to architecture for the Project?
• What is the Rough Order of Magnitude (ROM) Cost for mission outcomes?
• What is the current (spend) cost per goal?
• What was the actual obligations in prior year(s) (PY) for each Project?
• What Projects are budgeted for > $1M over their FYHSP? Projects spend will spend > $1M over lifetime?
• What are the Cost Drivers of Component mission Operations of Organization?
• What is Organization spending its money on IT investments in the upcoming year (in each component, etc.)?
• For each Project, what 'Year of Funds' are being used?
• For each Project, what resources will be used by fiscal year and appropriation?
• How does the requested dollar compare to their Budget Request or was appropriated during the year of execution for out-of-cycle?
• What is the trend Organization spending its money on IT investments (in each component, etc.)?
• What significant funding amount changes from prior year? Why?
• What is budget landscape; requested (pending), approved (certified), denied (unapproved)?
• What is out-of-cycle budget request landscape; pending, approved, unapproved?
• What would the impact be on the Component ('plan B') if the budget was reduced? How will the reduction affect Projects and portfolios?
• What is the mitigation/resolution(s) for each Project risk?
• Is the threat configuration traceable from management to device? For each threat managed Component applicable clearly documented through threat modeling, risk management, vendor specifications, integration testing and deployment (schedule)? Please identify the responsible party for each throughout the linkage. e.g. tracking sensors' ability to detect requirements may reside 'at a level' higher than the Project if the Project is viewed as a procurement.
• Any licensing/migration dates which are considered a risk, which products, and what is the go forward strategy? Are there problems with a vendor? A product to be supported in the near future?
• What are the Project's resulting risk mitigations?
• Does the Project maintain a Risk Register (RR)? Are mitigations defined and reviewed periodically? Is the Project appropriately managing & mitigating? Are the risk ratings improving over time, if not why?
• Does the Project have an updated Risk Management Plan (RMP)
• Have all changes to policies and/or regulations or business practices that require long lead times and impact on the Project been identified and included in the plans, and is the likelihood of such changes been included in the risk analysis?