# ODMDB Natural Language Query PoC

This is a **Proof of Concept (PoC)** that demonstrates the conversion of natural language queries into ODMDB search queries using OpenAI's structured output API.

## Current Status

⚠️ **Partial Implementation**: Currently only the **seekers** object mapping is implemented. This PoC focuses on demonstrating the natural language to DSL query conversion for seeker-related searches.

## Features

- **Natural Language Processing**: Converts human questions into structured ODMDB queries
- **Real ODMDB Integration**: Works with actual ODMDB data from `../smatchitObjectOdmdb/`
- **Schema-Based Mapping**: Uses actual seekers.json schema for accurate field mapping (62 properties)
- **Local Data Execution**: Processes queries against local seeker files in `objects/seekers/itm/`
- **OpenAI Structured Output**: Ensures reliable JSON query generation
- **Query Validation**: Validates generated queries against real ODMDB schema rules
- **jq Integration**: Powerful result processing, filtering, and CSV export capabilities

## Prerequisites

- Node.js (v16 or higher)
- OpenAI API key

## Installation

1. Make sure you have the ODMDB data structure available:

   ```
   ../smatchitObjectOdmdb/
   ├── schema/
   │   └── seekers.json       # Seeker schema (62 properties)
   └── objects/
       └── seekers/
           └── itm/           # Individual seeker JSON files
   ```

2. Install dependencies:

   ```bash
   npm install
   ```

3. Set your OpenAI API key:
   ```bash
   export OPENAI_API_KEY=sk-your-api-key-here
   ```

## Usage

### Running the PoC

**Query Generation Only (Default):**

```bash
npm start
```

**Query Generation + Execution:**

```bash
EXECUTE_QUERY=true npm start
```

````

This will process the hardcoded natural language query and output the generated ODMDB query in JSON format. When `EXECUTE_QUERY=true`, it will also execute the query against the ODMDB server.

### Changing the Query

To test different natural language queries, edit the `NL_QUERY` constant in `poc.js`:

```javascript
// Line 16 in poc.js
const NL_QUERY = "your natural language query here";
````

### Example Queries

**Status-based queries:**

- `"show me seekers with status startasap and their email and experience"`
- `"find seekers looking for jobs urgently with their skills and salary expectations"`
- `"get seekers who are not looking with their employment status"`

**Date-based queries:**

- `"give me new seekers since last week with email and experience"`
- `"show me seekers from yesterday with their location and availability"`
- `"find recently updated seekers with their job preferences"`

**Comprehensive field queries:**

- `"show me seeker contact info and work experience"`
- `"find seekers with personality types and language skills"`
- `"get seeker salary expectations and preferred working hours"`
- `"show me seeker education and training preferences"`
- `"find seekers with their job applications and saved jobs"`

**Location & preferences:**

- `"show me seekers in Paris with remote work preferences"`
- `"find seekers available to work in multiple countries"`
- `"get seekers with specific location and salary requirements"`

**Skills & competencies:**

- `"find seekers with technical skills and years of experience"`
- `"show me seekers with language abilities and personality profiles"`
- `"get seekers with specific know-how and job radar interests"`

**Job search activity:**

- `"show me seekers who applied to jobs recently"`
- `"find seekers with saved jobs and their preferences"`
- `"get seekers who were invited to apply with their status"`

**Notifications & communication:**

- `"show me seekers with email preferences and notification settings"`
- `"find seekers who receive weekly reports and interview tips"`

**Supported filter types:**

- **Status filtering**: `seekstatus` (startasap, norush, notlooking)
- **Date filtering**: `dt_create`, `dt_update`, `matchinglastdate` with date ranges
- **Index optimization**: Uses ODMDB indexes (`lst_alias`, `seekstatus_alias`) for efficient queries

### Demo & Testing Tools

**Interactive Demo:**

```bash
node demo.js
```

**Live PoC demonstration** that actually uses the query generation functionality to show:

- Real query generation from natural language using OpenAI
- ODMDB schema loading and field mapping
- Current ODMDB data status and sample data

**Demo with Query Execution:**

```bash
EXECUTE_DEMO=true node demo.js
```

Runs the demo with actual query execution against real seeker data files.

**jq Playground:**

```bash
node experiment-jq-playground.js
```

A playground to experiment with jq commands - not vital to the PoC but useful for learning jq syntax.

Demonstrates various jq operations including:

- Basic data formatting and field selection
- CSV conversion from JSON
- Advanced filtering and transformations
- Statistical summaries and aggregations

## Environment Variables

- `OPENAI_API_KEY` - Your OpenAI API key (required)
- `EXECUTE_QUERY` - Set to "true" to execute queries against ODMDB (default: false)
- `ODMDB_BASE_URL` - ODMDB server URL (default: http://localhost:3000)
- `ODMDB_TRIBE` - ODMDB tribe name (default: smatchit)
- `OPENAI_MODEL` - OpenAI model to use (default: gpt-5)

## Output Format

**Query Generation:**
The PoC generates ODMDB queries in this format:

```json
{
  "object": "seekers",
  "condition": ["prop.dt_create(>=:2025-10-06)"],
  "fields": ["alias", "email", "seekworkingyear"]
}
```

## ODMDB DSL Support

The PoC understands and generates these ODMDB DSL patterns:

- **Property queries**: `prop.<field>(operator:value)`
- **Index queries**: `idx.<indexName>(value)`
- **Join queries**: `join(remoteObject:localKey:remoteProp:operator:value)`

## Comprehensive Field Mappings

Supports extensive natural language mapping for **all 62 seeker properties**:

**Contact & Identity:**

- `email`, `contact`, `mail` → `email`
- `id`, `username`, `alias` → `alias`
- `bio`, `description`, `summary` → `shortdescription`

**Work Experience & Status:**

- `experience`, `years of experience`, `career length` → `seekworkingyear`
- `job titles`, `positions`, `roles`, `work history` → `seekjobtitleexperience`
- `status`, `availability`, `urgency` → `seekstatus`
- `employment`, `work status`, `job status` → `employmentstatus`

**Location & Geography:**

- `location`, `where`, `work location` → `seeklocation`
- `countries`, `work countries` → `countryavailabletowork`
- `current location`, `last location` → `lastlocation`

**Salary & Compensation:**

- `salary`, `pay`, `compensation`, `wage` → `salaryexpectation`
- `currency`, `salary currency` → `salarydevise`
- `salary unit`, `pay period` → `salaryunit`

**Skills & Competencies:**

- `skills`, `competencies`, `abilities` → `skills`
- `languages`, `language skills` → `languageskills`
- `knowledge`, `expertise`, `know-how` → `knowhow`

**Personality & Preferences:**

- `personality`, `MBTI`, `type` → `mbti`
- `likes`, `interests`, `preferences` → `thingsilike`
- `dislikes`, `avoid`, `not interested` → `thingsidislike`

**Job Search Activity:**

- `applied jobs`, `applications` → `jobadapply`
- `saved jobs`, `bookmarked jobs` → `jobadsaved`
- `viewed jobs`, `job views` → `jobadview`
- `invitations`, `invited to apply` → `jobadinvitedtoapply`

**Availability & Schedule:**

- `working hours`, `preferred hours`, `schedule` → `preferedworkinghours`
- `unavailable`, `blocked times` → `notavailabletowork`

**Dates & Activity:**

- `created`, `new`, `recent`, `since` → `dt_create`
- `updated`, `modified`, `last update` → `dt_update`
- `last matching`, `matching date` → `matchinglastdate`

_Plus comprehensive mappings for education, notifications, training, and system fields._

## Schema Context

The PoC can optionally load schema files for context:

- `main.json` - Combined schema definitions
- `lg.json` - Localization/language mappings

## Limitations

- **Seekers only**: Other ODMDB objects (jobads, recruiters, etc.) are not yet implemented
- **Local execution only**: Works with file-based data, not live ODMDB server API
- **Hardcoded query**: Single query per run (no interactive mode)
- **Performance limit**: Processes first 50 seeker files for PoC performance
- **Simplified DSL**: Basic condition parsing (date ranges, status filtering)

## Next Steps

- [ ] Add support for other ODMDB objects (jobads, recruiters, etc.)
- [ ] Interactive CLI for multiple queries
- [ ] Integration with actual ODMDB backend
- [ ] Enhanced field mapping and validation
- [ ] Multi-turn conversation support

## Files

**Core Implementation:**

- `poc.js` - Main PoC implementation with full ODMDB integration
- `package.json` - Dependencies and scripts

**Demo & Testing:**

- `demo.js` - **Live PoC demo** that actually generates and executes queries using real ODMDB data
- `experiment-jq-playground.js` - jq learning playground (optional, not vital to PoC)

**Data & Schema:**

- `main.json` - Optional consolidated schema context (if available)
- `../smatchitObjectOdmdb/schema/seekers.json` - Real seekers schema (62 properties)
- `../smatchitObjectOdmdb/objects/seekers/itm/` - Individual seeker data files