prompt(core): add cluster health check #434

ropatil010 · 2025-11-03T18:05:36Z

Hi Team,

Can you PTAL on this.

PR about:
Implements comprehensive cluster health check tool that examines:

Cluster operators (OpenShift)
Nodes (readiness, schedulability, resource pressure)
Pods (failures, crash loops, image pull errors, high restarts)
Workload controllers (Deployments, StatefulSets, DaemonSets)
Storage (PVC status)
Recent warning events

Features:

Text and JSON output formats
Verbose mode for detailed diagnostics
Configurable event checking
Clear severity indicators (critical/warning/healthy)

ropatil010 · 2025-11-03T18:07:55Z

Output format:

===============================================
Cluster Health Check Report

Cluster Type: OpenShift
Cluster Version: 4.20.0
Check Time: 2025-11-03T17:33:34Z

Checking Cluster Operators...
✅ All cluster operators healthy (34/34)
Checking Node Health...
✅ All nodes healthy (6)
Checking Pod Health...
❌ CRITICAL: 1 pod(s) in failed/pending state

openshift-xxxx/nodename [Failed]
Checking Workload Controllers...
✅ All deployments healthy
✅ All statefulsets healthy
✅ All daemonsets healthy
Checking Storage...
✅ All PVCs bound
Checking Recent Events...
✅ No recent warning events

===============================================
Summary

Critical Issues: 1
Warnings: 0

❌ Cluster has CRITICAL issues requiring immediate attention

pkg/toolsets/core/health_check.go

manusa · 2025-11-05T07:59:31Z

I'm not really sure about this one, it seems to be quite opinionated.
Given the raw tools, the LLMs should be able to perform the operations themselves.

In https://github.com/Flux159/mcp-server-kubernetes they're implementing similar functionality via a prompt. Which doesn't seem like a bad idea.

Similarly, in https://github.com/GoogleCloudPlatform/kubectl-ai (AFAIU) they're leveraging the system prompt for this purpose: https://github.com/GoogleCloudPlatform/kubectl-ai/blob/main/pkg/agent/systemprompt_template_default.txt

IMO if we want to proceed with this feature we should either:

Reimplement it as a prompt for the core toolset.
If we really want this as a tool, add it to a specific opt-in toolset diagnostics or something similar.

This is also a good case to test evals and see if they can be used to make a better decision on how to implement this feature.

ropatil010 · 2025-11-07T15:28:56Z

Hi @manusa/@matzew Updated as per suggestions.
Can you PTAL when ever get a chance. Thanks in adv.!

manusa

I managed to break down prompt support into:

#226
#556

This PR is blocked by #556 that should be taken care of quickly (since there's already a PR that partially takes care of it #510)

Once that's merged we can rebase this one and properly review.

nader-ziada · 2025-12-12T17:24:50Z

@ropatil010 would be you be able to rebase this PR? thanks

ropatil010 · 2025-12-15T12:44:32Z

Hi @nader-ziada , i have rebase the PR and updated the files accordingly by using the existing function definitions.
Can you PTAL?

pkg/mcp/mcp.go

nader-ziada · 2025-12-15T15:03:18Z

Hi @nader-ziada , i have rebase the PR and updated the files accordingly by using the existing function definitions. Can you PTAL?

sorry, you will have to rebase again, but should be a small conflict this time, please see comment, thanks

nader-ziada · 2025-12-15T18:41:01Z

thanks @ropatil010

nader-ziada · 2025-12-15T20:36:32Z

@ropatil010 The TestNotifiesToolsChangeMultipleTimes is failing, check for a fix for a similar issue in TestNotifiesToolsChange

nader-ziada · 2025-12-15T20:43:26Z

@manusa any comments about this PR?

ropatil010 · 2025-12-16T13:59:23Z

Hi @manusa/@nader-ziada Fixed as per suggestions. Can you PTAL when ever get a chance?
Thanks in advance.

nader-ziada · 2025-12-17T20:24:55Z

hi @ropatil010 everything looks good, but looks like all the refactoring going on currently makes this PR need another rebase, check Marc comment here as well: #591 (comment)

I can do the rebase for you if you don't get a chance to do it.

Signed-off-by: Rohit Patil <ropatil@redhat.com>

…ontainers#589 Signed-off-by: Rohit Patil <ropatil@redhat.com>

ropatil010 · 2025-12-18T06:54:49Z

Hi @nader-ziada , Thanks for reviewing. Updated the files/rebased as per the new design. Can you PTAL when ever get a chance?
Thank you!

manusa

Hi @ropatil010, thanks for working on this.

In general, you can now use the standard client-sets which will make the code more resilient and readable.

Otherwise, this seems OK to go.

I'm not sure if you've attempted to provide other kinds of output besides Markdown or mixing some sturctured output within the markdown itself (e.g. resources enclosed within blocks, etc.)
I guess that at some point we can add some evals to benchmark results based on different output formats

pkg/toolsets/core/toolset.go

pkg/toolsets/core/health_check.go

- Remove verbose argument, add klog progress logging - Use CoreV1 clientset directly for cleaner code - Extract prompt initialization to initHealthChecks() Signed-off-by: Rohit Patil <ropatil@redhat.com>

Replace unstructured resource access with typed CoreV1 and AppsV1 clientsets for improved type safety and code clarity. Signed-off-by: Rohit Patil <ropatil@redhat.com>

docs/PROMPTS.md

Signed-off-by: Rohit Patil <ropatil@redhat.com>

manusa

LGTM, thx!

ropatil010 force-pushed the cluster-health-check branch 2 times, most recently from 2e02dc9 to 767c1db Compare November 4, 2025 05:23

Cali0707 requested a review from manusa November 4, 2025 15:39

matzew reviewed Nov 5, 2025

View reviewed changes

pkg/toolsets/core/health_check.go Outdated Show resolved Hide resolved

ropatil010 force-pushed the cluster-health-check branch 2 times, most recently from 5fd5987 to afb7d63 Compare November 5, 2025 16:03

ropatil010 changed the title ~~feat(core): add cluster health check tool~~ prompt(core): add cluster health check Nov 7, 2025

ropatil010 force-pushed the cluster-health-check branch from b7fde2e to 8b8c9e9 Compare November 17, 2025 15:12

This was referenced Dec 5, 2025

[API] Toolset-defined prompt templates #556

Closed

feat: add MCP prompts support #510

Closed

manusa reviewed Dec 5, 2025

View reviewed changes

ropatil010 force-pushed the cluster-health-check branch 2 times, most recently from 6a8f524 to 17bda66 Compare December 15, 2025 11:26

ropatil010 changed the title ~~prompt(core): add cluster health check~~ [WIP] prompt(core): add cluster health check Dec 15, 2025

ropatil010 force-pushed the cluster-health-check branch from 17bda66 to 7a56109 Compare December 15, 2025 12:41

ropatil010 changed the title ~~[WIP] prompt(core): add cluster health check~~ prompt(core): add cluster health check Dec 15, 2025

nader-ziada reviewed Dec 15, 2025

View reviewed changes

pkg/mcp/mcp.go Show resolved Hide resolved

ropatil010 force-pushed the cluster-health-check branch from 7a56109 to 8302c9f Compare December 15, 2025 17:11

manusa mentioned this pull request Dec 17, 2025

test(prompts): split notification tests to assert specific list changes #591

Merged

ropatil010 added 4 commits December 18, 2025 10:29

feat: add cluster health check prompt

ac82576

Signed-off-by: Rohit Patil <ropatil@redhat.com>

test(core): fix cluster health check prompt test

30b29f1

Signed-off-by: Rohit Patil <ropatil@redhat.com>

test: accept prompts change notifications in watch tests

75b285a

Signed-off-by: Rohit Patil <ropatil@redhat.com>

fix: align tests with PR containers#591 and update to new API from PR c…

c3f3217

…ontainers#589 Signed-off-by: Rohit Patil <ropatil@redhat.com>

ropatil010 force-pushed the cluster-health-check branch from 5dc66c5 to c3f3217 Compare December 18, 2025 06:22

manusa requested changes Dec 18, 2025

View reviewed changes

ropatil010 added 2 commits December 18, 2025 21:48

refactor(core): add logging and improve health check API

60c6270

- Remove verbose argument, add klog progress logging - Use CoreV1 clientset directly for cleaner code - Extract prompt initialization to initHealthChecks() Signed-off-by: Rohit Patil <ropatil@redhat.com>

refactor(core): use typed clientsets in health check diagnostics

cea59e4

Replace unstructured resource access with typed CoreV1 and AppsV1 clientsets for improved type safety and code clarity. Signed-off-by: Rohit Patil <ropatil@redhat.com>

ropatil010 force-pushed the cluster-health-check branch from 0304e3f to cea59e4 Compare December 19, 2025 05:15

nader-ziada reviewed Dec 19, 2025

View reviewed changes

docs/PROMPTS.md Outdated Show resolved Hide resolved

docs: fix cluster health check prompt arguments

5a5a0e2

Signed-off-by: Rohit Patil <ropatil@redhat.com>

manusa added this to the 0.1.0 milestone Dec 23, 2025

manusa approved these changes Dec 23, 2025

View reviewed changes

manusa merged commit a153e2f into containers:main Dec 23, 2025
6 checks passed

prompt(core): add cluster health check #434

prompt(core): add cluster health check #434

Conversation

ropatil010 commented Nov 3, 2025

Uh oh!

ropatil010 commented Nov 3, 2025

=============================================== Cluster Health Check Report

=============================================== Summary

Uh oh!

Uh oh!

manusa commented Nov 5, 2025

Uh oh!

ropatil010 commented Nov 7, 2025

Uh oh!

manusa left a comment

Choose a reason for hiding this comment

Uh oh!

nader-ziada commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ropatil010 commented Dec 15, 2025

Uh oh!

Uh oh!

nader-ziada commented Dec 15, 2025

Uh oh!

nader-ziada commented Dec 15, 2025

Uh oh!

nader-ziada commented Dec 15, 2025

Uh oh!

nader-ziada commented Dec 15, 2025

Uh oh!

ropatil010 commented Dec 16, 2025

Uh oh!

nader-ziada commented Dec 17, 2025

Uh oh!

ropatil010 commented Dec 18, 2025

Uh oh!

manusa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

manusa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

===============================================
Cluster Health Check Report

===============================================
Summary

nader-ziada commented Dec 12, 2025 •

edited

Loading