r/devops 14h ago

Architecture Where is AI actually adding value in your DevOps workflow and where isn't it?

0 Upvotes

Senior DevOps here. AI is being pushed into every part of the pipeline right now, and I want to cut past the hype and hear real production experience.

Two questions:

Where have you put AI into production in your DevOps process and it genuinely adds value? (e.g. CI/CD, code review, IaC generation, monitoring/alerting, incident response, log analysis, documentation)

Where did it not prove worthwhile? Think cost, alert noise, false positives, or maintenance overhead that outweighed the benefit.

Thanks.


r/devops 7h ago

Discussion I am a Sr System Administrator and want to switch to DevSecOps

3 Upvotes

To begin with this is my first post or i don't know what it is called on reddit,

Apologies for the mistakes in my English please ignore it,

But I really want to switch from system admin job to DevSecOps

About myself

I have 12 years of experience starting from desktop support Engineer to laptop repairing to Data center to Monitoring to Asset management to system administration, I have knowledge of both windows and Linux operating system and currently managing Windows and Linux server (QA) no prod since my senior have never let me work on prod server even though I have worked with him since 7 years, handled a US based client for 3 years working with his team and currently handling two clients one with just some basic needs regarding systems and O365 and second with Linux servers, kind of devops but not fully the second client have a deployment using jenkins ( fetch the code from got repo, call the specific server through ssh and run the deployment and build script)

Worked with a friend's company as a contractor on his client for 6 months which ended last December and since then no work on core devops, started learning k8s and currently learning it but not able to give 100% to it also leaning python scripting and bash scripting by the help or AI can read the code and understand what it does but can not write with full confidence

I know the tools below tools

Terraform

Github

Github pipeline

Jenkins ( basic free style pipeline)

Got the idea of Sonarqube while working as a contractor but it was short lived

Understanding New relic as now the client wants to setup it

AWS basic

Azure basic and ADO basic

GCP not so much

Done a course in cybersecurity as well from local institute but not able to perform handson

I honestly want to work in Devsecops

I am already 38 and according to me not earning enough (peer pressure)

Any suggestions any help or direction is appreciated

I am even ready to do an unpaid internship ( beside the job since i have responsibilities)

Call it desperation, determination or pressure but I really want to work in Devsecops domain

Please help


r/devops 10h ago

Ops / Incidents How do you handle production patching for EC2 instances?

0 Upvotes

how do you handle production patching for EC2 instances in your environment?

I'm interested in learning about real-world production practices.

Some questions:

  • Do you use AWS Systems Manager Patch Manager, Patch Policies, or another approach?
  • Do you patch EC2 instances in place, or do you replace them with new AMIs (immutable infrastructure)?
  • How do you schedule maintenance windows and minimize downtime?
  • How do you handle Auto Scaling Groups during patching?
  • What's your rollback strategy if a patch causes issues?
  • Do you test patches in dev/staging before production?
  • How much of the process is automated versus manually approved?

I'd really appreciate hearing how your organization handles production patching at scale, along with any best practices or lessons learned.


r/devops 9h ago

Ops / Incidents At what point do you stop treating timeouts as normal?

0 Upvotes

Hey guys! we've got a couple of internal services that will occasionally throw timeout errors under load. They retry, recover, and users never seem to notice.

The annoying part is that it's been happening long enough that nobody really reacts to it anymore. Every incident review ends up with the same general response that it's just something we've always seen.

Now I'm wondering if thats just the reality of distributed systems, or if we've gotten too comfortable ignoring something that probly deserved a closer look a while ago.

How do you decide when intermittent timeouts have crossed the line from something you live with to something worth digging into?


r/devops 19h ago

Observability Good observability tool recommendation - Cloud Based

0 Upvotes

I work in two companies, one very big that have a giant budget to spend on Datadog, and another small one, that don't have that very big budget.

On the small company we are looking to migrate from Prometheus + Grafana + Alert Manager to something cloud based. Mostly because we are a small team (3 people only) and we don't have time to spend maintaining the infrastructure for it to run.

Now, is there some good alternative to Datadog? Datadog is the king, but is brutally expensive.

I've tried Signoz, looked promising, but is bad. They documentation is bad, there are just general ideas without details (they assume that you know a lot from I don't know where), they support is lame (they only have a chat to "Interact with a Human", that takes days to receive an answer, and their UI is buggy. Really bad experience.

But looking there, they are the only ones having like "close" experience to Datadog. We tried New Relic in the past, but they have all metrics and log scattered over the place without correlation. Also tried BetterStack but lacks a lot of features compared to other things.

So, is there some good observability platform cloud-based that I'm not aware of, outside Datadog? I'm not completely closed to self-hosting as long as it takes low effort to maintain.


r/devops 13h ago

Discussion Who owns the "why we did it this way" knowledge on your team?

0 Upvotes

Small team here and I keep running into the same thing. The reasoning behind half our setup lives in two people's heads.

Not in the runbooks, not in Confluence, not in the repo. Someone new joins and rediscovers everything the hard way and when one of those two is out we just stall.

It got noticeably worse once we started leaning on Claude Code and Cursor for real work. The agents read whatever context files we give them and treat it as truth, so now stale knowledge doesn't just slow a person down, it gets confidently baked into code by something that has no idea the decision changed months ago.

So I wonder how this actually works at bigger shops. Is there anyone whose job includes keeping that stuff current or does it just live with whoever happens to remember? And has anyone here had a documentation process survive past the first busy quarter or does it always quietly die?

Not looking for tool recommendations just trying to work out whether we're unusually bad at this or whether it's the normal state of things.


r/devops 9h ago

Career / learning Best way to learn K8 now?

26 Upvotes

Hi all, I’m learning Kubernetes right now for work (I use Colima + kubectl) - what is the best way to do so with AI?

I’ve used docker before in previous projects and studied concepts of containers in school - im no expert but I should know enough to learn Kubernetes.

I started the Udemy + kodekloud course: Kubernetes for the absolute beginners - Hands on. I’m not sure if it’s relevant for 2026 - so far it’s asking me to manually write yaml files, which I’m sure is important, but do I really need to be able to write yaml files with vscode extensions and AI that writes it for me?

So basically, what’s the best way to learn Kubernetes in 2026? I would appreciate any courses I should complete after my current one or a roadmap.

I have around a month or so to be good enough to collaborate with our platform and aws teams.


r/devops 15h ago

Architecture Multi platform build

1 Upvotes

I am planning to have my docker images to be multi architecture and for it my plan is to have two codebuilds for arm and amd when they both succeeds event bridge will invoke a lambda and lambda will merge the images into one

But I am unable to think a way to set up event my current plan is something like when codebuilds are triggered if anyone succeeds let's say arm build then event bridge will match the source version and and if the other build also success then it will invoke lambda

Apparently there is no ability to compare source version in eventbridge and I would have to invoke lambda and lambda has to compare the source version

Is there any way I can implement this


r/devops 14h ago

Discussion Currently on Falco for runtime security — anyone moved to Tetragon/KubeArmor/Tracee and regretted (or loved) it?

2 Upvotes

Running Falco in our EKS clusters right now for runtime detection, paired with Kyverno for admission control and Cosign/Vault for supply chain signing/secrets. Been solid so far, but I keep seeing Tetragon and KubeArmor come up as alternatives, especially for teams who want enforcement (block, not just alert) rather than detect-and-notify.

Curious what people are actually running in production and why:

  • If you moved off Falco, what pushed you? Overhead, rule fatigue, lack of enforcement?
  • Anyone running Tetragon specifically for the eBPF enforcement piece? Is it worth the Cilium tie-in if you're not already on Cilium for CNI?
  • KubeArmor folks: how's LSM behavior in practice across different node kernel configs (AppArmor vs BPF-LSM)? Heard that's where it gets messy.
  • Tracee: anyone using this seriously in prod or is it mostly a DFIR/forensics tool for you?

Not looking to rip out Falco, just trying to figure out if there's a compelling reason to add enforcement on top, or if pairing Falco with Falco Talon covers that gap well enough.