Data & Knowledge Navigator
What information do you have, what do you want to accomplish with it, and what must happen before AI can use it effectively?
The Data & Knowledge Navigator guides you through information preparation, quality, representation, retrieval, analysis, and governance.
Unlike AI Solver, which starts with a task, this dimension starts with your information and its requirements.
Explore Representation and Retrieval Methods
Which representation, learning method, or model mechanism should I investigate?
Start with AI Algorithms & Learning Methods Navigator, or choose a specific branch:
- Choose an information representation
- Compare retrieval and knowledge-matching methods
- Investigate statistical assumptions and relationships
Start: What Is Your Information Challenge?
| Your Situation | Follow This Branch |
|---|---|
| I need to collect or organize information. | A. Identify and Structure Data |
| My data may be incomplete, incorrect, or inconsistent. | B. Improve Data Quality |
| I need AI to understand or compare information. | C. Represent Information |
| I want answers based on documents or databases. | D. Retrieve Knowledge |
| I need to discover patterns or relationships. | E. Analyze Information |
| My information has privacy, security, or governance requirements. | F. Govern Information |
A. Identify and Structure Data
Question: What type of information do you have?
- Rows, columns, numbers, or transactions: Data Science → Database → Data Preprocessing
- Documents and written language: Document Understanding AI → Natural Language Processing (NLP) → Context
- Images and video: Vision → Video/Image → Feature Exploration/Learning
- Audio and speech: Speech Recognition → Signal Processing → End-to-End Speech
- Entities with important relationships: Graph → Knowledge Graphs
- Information arriving from connected devices: Internet of Things (IoT) → Digital Twin → Data Interoperability
Next decision: Is the information ready to use?
- No or uncertain: Check its quality and preparation requirements →
- Yes: Choose an appropriate representation →
B. Improve Data Quality
Question: What is wrong with the data?
- Missing, inconsistent, or invalid values: Data Quality → Data Preprocessing
- Potentially misleading or unrepresentative observations: Bias and Variances → Datasets
- Features are difficult to interpret or select: Feature Exploration/Learning
- Information comes from incompatible systems: Data Interoperability
- The source or correctness is uncertain: Data Governance → AI Verification and Validation
Decision: Can the problem be corrected without destroying important information?
- Yes: Document the correction and validate the resulting dataset using Data Quality and Evaluation.
- No: Improve collection, investigate the source, or revise the intended use before building the AI system.
Checkpoint: Can you explain where the data originated, what it represents, and its known limitations?
Continue: Choose its representation →
C. Represent Information
Question: How must AI use the information?
- Predict from defined variables: Feature Exploration/Learning → Representation Learning
- Compare information by semantic similarity: Embedding → Semantic Search
- Work with relationships among entities: Knowledge Graphs → Graph
- Reduce the complexity of many features: Dimensional Reduction → Principal Component Analysis (PCA)
- Preserve structured business records: Database → Data Interoperability
- Maintain information across tasks: Memory → Context
Decision: Does the system need explicit relationships or primarily similarity-based retrieval?
- Explicit relationships: Explore Knowledge Graphs.
- Similarity between passages or objects: Explore Embedding and Semantic Search.
- Both: Explore a combined retrieval architecture using Retrieval-Augmented Generation (RAG) and Knowledge Graphs.
Continue: Retrieve information → or Analyze it →
D. Retrieve Knowledge
Question: Where are the answers located?
- A small collection that fits in model context: Context → In-Context Learning (ICL)
- A large or changing document collection: Semantic Search → Retrieval-Augmented Generation (RAG)
- Structured records requiring exact values: Database → Data Interoperability
- Connected entities and relationships: Knowledge Graphs
- Information spread across multiple systems: LlamaIndex → LangChain → Agents/Assistants
Decision: Do answers need supporting evidence?
- Yes: Preserve source references and evaluate retrieval and answer quality using AI Verification and Validation.
- No: You should still test factual correctness and identify the limitations of the selected approach.
Checkpoint: Can the system find the right source and produce an answer supported by that source?
Continue: Govern and validate the information →
E. Analyze Information
Question: What do you want to discover?
- Relationships or trends: Analytics → Visualization
- Groups of similar observations: Clustering
- Unusual events or values: Anomaly Detection
- What influences an outcome: Feature Exploration/Learning → Causation vs. Correlation
- Relationships in a network: Social Network Analysis (SNA) → Graph
- Future values or behavior: Predictive Analytics → Forecasting
Decision: Are you describing existing information or predicting something new?
- Describing: Explore Analytics and Visualization.
- Predicting: Continue to AI Solver for model selection.
- Establishing causes: Investigate Causation vs. Correlation before interpreting associations as causal effects.
F. Govern Information
Question: What constraints apply to the information?
- Privacy or personal data: Privacy → Data Governance
- Security or unauthorized access: Cybersecurity → Data Governance
- Legal or organizational obligations: Law → Risk, Compliance and Regulation
- Quality and traceability: Data Quality → AI Verification and Validation
- Information shared across organizations: Data Interoperability → Data Governance
- Sensitive information sent to external AI services: Privacy → AI Governance
Decision: Can the intended use meet the applicable requirements?
- Yes: Document controls and continue with testing.
- Not yet: Restrict access, minimize information exposure, or redesign the workflow.
- Uncertain: Obtain appropriate privacy, security, or legal review before deployment.
Continue: Dimension 7: AI Trust & Evaluation →
Complete Your Information Path
Before selecting or deploying AI, verify that you can answer:
- What information is available?
- Is it fit for the intended purpose?
- How should it be represented?
- How will the system find relevant information?
- How will correctness, privacy, and provenance be managed?
| Next Question | Decision Tree |
|---|---|
| What AI approach fits my information and task? | AI Solver |
| How do I implement the system? | AI Builder |
| Can I trust the resulting system? | AI Trust & Evaluation |
| How are these representations processed? | How AI Works |