Skip to content

[WIP] HIVE-29370: Text-to-SQL with HMS MCP Server#6421

Open
deniskuzZ wants to merge 2 commits intoapache:masterfrom
deniskuzZ:hms-mcp-server
Open

[WIP] HIVE-29370: Text-to-SQL with HMS MCP Server#6421
deniskuzZ wants to merge 2 commits intoapache:masterfrom
deniskuzZ:hms-mcp-server

Conversation

@deniskuzZ
Copy link
Copy Markdown
Member

@deniskuzZ deniskuzZ commented Apr 10, 2026 •
edited
Loading

What changes were proposed in this pull request?

MCP Server that exposes Hive Metastore metadata via the Iceberg REST Catalog API.

Why are the changes needed?

Expose Data Lake metadata through MCP

Does this PR introduce any user-facing change?

No

How was this patch tested?

pip install -r $HIVE_HOME/scripts/nlsql/requirements.txt

mvn clean install -DskipTests -T8 -Pdist -Dmaven.javadoc.skip -Drat.skip --fail-at-end

export HIVE_VERSION=4.3.0-SNAPSHOT
docker-compose --profile llap down --rmi all
./build.sh -hive ${HIVE_VERSION} -hadoop 3.4.2 -tez 0.10.5
./start-hive.sh --llap

export ANTHROPIC_BASE_URL="https://api.anthropic.com"   # or your gateway URL
export ANTHROPIC_AUTH_TOKEN="your-token"                 # or use ANTHROPIC_API_KEY
export ANTHROPIC_MODEL="anthropic.claude-opus-4-6"       
export MCP_SERVER_URL="http://localhost:3000/sse"

$HIVE_HOME/bin/beeline -u "jdbc:hive2://localhost:10000"

beeline> !nlsql show me the top 10 orders by amount
beeline> !nlsql count all rows in the customers table
beeline> !nlsql which tables have more than 1 million rows

@deniskuzZ deniskuzZ changed the title [WIP] HMS MCP Server [WIP] HIVE-29370: HMS MCP Server Apr 10, 2026
@deniskuzZ deniskuzZ changed the title [WIP] HIVE-29370: HMS MCP Server [WIP] HIVE-29370: Text-to-SQL with HMS MCP Server Apr 10, 2026
@Aggarwal-Raghav
Copy link
Copy Markdown
Contributor

Aggarwal-Raghav commented Apr 10, 2026

I initally thought the implemenation will be in java using spring-ai 😅

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Apr 10, 2026

Quality Gate Passed Quality Gate passed

Issues
19 New issues
0 Accepted issues

Measures
1 Security Hotspot
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@dengzhhu653 dengzhhu653 self-requested a review April 11, 2026 01:46
zhangbutao
zhangbutao reviewed Apr 11, 2026
View reviewed changes
Comment thread standalone-metastore/metastore-tools/mcp-server/metastore_mcp_server.py
"""
Metastore MCP Server — exposes Hive Metastore metadata via Model Context Protocol.

Backed by the Metastore Iceberg REST Catalog API. Provides tools for LLM agents
Copy link
Copy Markdown
Contributor

@zhangbutao zhangbutao Apr 11, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only Iceberg tables can be supported?
What about other external tables? such as jdbc tables?

difin
difin reviewed Apr 12, 2026
View reviewed changes
Comment thread beeline/src/java/org/apache/hive/beeline/Commands.java
String scriptPath;
if (hiveHome != null) {
scriptPath = hiveHome + File.separator + "scripts" + File.separator +
"nlsql" + File.separator + "nlsql_agent.py";
Copy link
Copy Markdown
Contributor

@difin difin Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be more readable with String.format(...).

difin
difin reviewed Apr 12, 2026
View reviewed changes
Comment thread beeline/src/java/org/apache/hive/beeline/Commands.java
Comment on lines +1212 to +1213
scriptPath = "scripts" + File.separator + "nlsql" + File.separator +
"nlsql_agent.py";
Copy link
Copy Markdown
Contributor

@difin difin Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

difin
difin reviewed Apr 12, 2026
View reviewed changes
Comment thread beeline/src/java/org/apache/hive/beeline/Commands.java
String DIM = "\u001B[2m";
String RESET = "\u001B[0m";

beeLine.output(DIM + "Generating SQL for: " + RESET + CYAN + nlQuery + RESET);
Copy link
Copy Markdown
Contributor

@difin difin Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

difin
difin reviewed Apr 12, 2026
View reviewed changes
Comment thread beeline/src/java/org/apache/hive/beeline/Commands.java
Comment on lines +1283 to +1291
// Read stderr (errors/warnings)
StringBuilder errBuilder = new StringBuilder();
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(process.getErrorStream()))) {
String errLine;
while ((errLine = reader.readLine()) != null) {
errBuilder.append(errLine).append("\n");
}
}
Copy link
Copy Markdown
Contributor

@difin difin Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These lines are identical to lines 1270-1281, should we have a helper method to reduce code duplication?

@difin
Copy link
Copy Markdown
Contributor

difin commented Apr 12, 2026

There are 19 sonar issues https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6421&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true

@deniskuzZ
Copy link
Copy Markdown
Member Author

deniskuzZ commented Apr 15, 2026 •
edited
Loading

Thanks everyone for the review and feedback!
This was just a prototype idea for an intern project, and it’s great to see @hemanthumashankar0511 taking interest.
It’s your playground now - feel free to use this PR as inspiration and explore any direction you find interesting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhangbutao zhangbutao zhangbutao left review comments

@difin difin difin left review comments

@dengzhhu653 dengzhhu653 Awaiting requested review from dengzhhu653

Assignees

No one assigned

Labels

tests passed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@deniskuzZ @Aggarwal-Raghav @difin @zhangbutao @asf-ci-hive