#OpenTelemetry
Event report of yesterday's talk "Best Practices for using Amazon Distro for #opentelemetry
https://last9.hashnode.dev/event-report-best-practices-for-using-amazon-distro-for-opentelemetry
Today’s the day! Don't miss Iris Dyrmishi on #OTel in Practice, hosted by the OTel End-User Working Group.
When: June 8th at 10:00 PT/13:00 ET/19:00 CET
Calendar invite: https://shorturl.at/mvyBZ
Starting in an hour
"Best Practices for using Amazon Distro for #OpenTelemetry" with Michael Hausenblas
Join here: https://discord.gg/hr3tCQ4prM?event=1108408359272595467
Join me at @\jcon_conference on June 20th for an introduction to #OpenTelemetry tracing for #developers
https://jconeurope2023.sched.com/event/1K40F/practical-introduction-to-opentelemetry-tracing
today started with me innocently asking myself "but how DOES #python #OpenTelemetry auto-instrumentation actually work?!"
and ended with me buying a CPython Internals book
📖
Good morning #Coimbra! If you're at #JNation and don't have plans after the coffee break, why not consider listening to #Observability related topics. Check my presentation at the Mondego room at 16:30! We'll talk about #Java #Micrometer #OpenZipkin #OpenTelemetry and more!
Happening this week! Don't miss Iris Dyrmishi on #OTel in Practice, hosted by the OTel End-User Working Group.
When: June 8th at 10:00 PT/13:00 ET/19:00 CET
Calendar invite: https://shorturl.at/mvyBZ
🔥New Video This Week🔥
📚Dive into the world of distributed tracing with Opentelemetry and Spring Boot
➡️Code walkthrough to understand #Opentelemetry configuration
➡️Explore traces using Jaeger UI
tracepusher v0.6.0 has been released.
tracepusher v0.6.0:
- Adds support for span attributes
- Adds documentation website: https://agardnerit.github.io/tracepusher/
- Both #opensource and #dynatrace Killercoda tutorials have been upgraded
Missed last week’s OTel Q&A? No problem! You can catch the video recording on YouTube!👇
Thanks Iris Dyrmishi for sharing her story with us!
https://www.youtube.com/watch?v=9iaGG-YZw5I&feature=youtu.be
OpenSearch and OpenTelemetry on Kubernetes:
https://medium.com/@chfrank_cgn/deconstructing-opensearch-efb9d20cd667
Happy Monday, y'all! In case you missed #OSSUMMIT earlier this month in Vancouver, you can now catch a recording of our panel discussion on fostering community in #OpenTelemetry. Featuring @austinlparker, @reesesbytes, fellow #cncfAmbassador @cyberynn, and me.
Video overview now on YT: Tracing #gitlab pipelines with #opentelemetry https://youtu.be/zZDFQNHepyI
If you're in the APAC region or are a night owl in the Americas region, you can catch me on June 1st at 11:00 SGT (23:00 ET/20:00 PT) for the DevOps Institute's SkillUp Hours, where I'll be talking about #OpenTelemetry and #Observability, alongside Suresh GP and BMK Lakshminarayanan! !
Register here👇
https://buff.ly/45yvKCv
Announcing #Azure Monitor #OpenTelemetry Distro https://devblogs.microsoft.com/dotnet/azure-monitor-opentelemetry-distro/
Q: When I am creating a new client library, should I add #opentelemetry support from scratch?
A: Yes. aber naturlich!
Olá 👋! Will you be attending #JNation? So will I 💪🏾! I'll be talking about #observability #Micrometerio #SpringBoot #OpenTelemetry and more! Check the conference page at https://buff.ly/3LEOWW2 #java
done for the week:
* #elastic vector search meetups in vienna and barcelona
* another amazing #SpringIO23 with an #opentelemetry workshop + booth
* also speaker 🚲🍇🫒 today
In case you missed @Rejekts last month in Amsterdam, you can now catch a recording of our panel discussion on fostering community in #OpenTelemetry. Moderated by @austinlparker, and featuring @reesesbytes, fellow #cncfAmbassador @cyberynn, and me.
It’s always huge fun to watch one of your favourite podcasts recording live 😀
@noctovis @richcampbell @carlfranklin discussing #OpenTelemetry at #Techorama
Send them a toot! 😀
@Martindotnet I have a question about instrumenting code for #OpenTelemetry. When instrumenting a c# system that relies heavily in messages via topics/queues, do libraries like #MassTransit or #Wolverine already do the traceid and baggage serializing and deserializing or is this something that doesn’t come out-of-the-box? Thanks!
This week on Slight Reliability Bruce Cullen is back!
We chat about his experience at #kubeconeu and topics such as:
- Does #OpenTelemetry live up to the hype?
- Is #GreenEngineering a future career path for performance engineers?
- What is eBPF and cilium?
...and much more.
If you've never been to KubeCon before, Bruce gives a great explanation of what it was like, how it was structured, what it's all about, etc.
It was great being back! Here are the slides of my keynote today at CloudConf:
From complexity to observability using OpenTelemetry
https://speakerdeck.com/danilop/from-complexity-to-observability-using-opentelemetry
This week! The OTel End User Discussions are happening! Come chat about all things #OpenTelemetry:
✨ EMEA: May 16, 11AM CET (GMT +1)
✨ APAC: May 17, 11AM India ST (GMT +5.5)
✨ AMER: May 18, 9AM PT (GMT -8)
Find Zoom info here:
https://opentelemetry.io/community/end-user/discussion-group/
Threw together a quick and dirty sample showing ASP.NET Core observability with Prometheus, Loki, Grafana, and thje Opentelemetry Collector
https://github.com/cecilphillip/grafana-otel-dotnet
#dotnet #opentelemetry #grafana
🎶do you want to build an exporter?
🎶write some go and hack all day?
https://github.com/equinix-labs/otel-cli/pull/205
#opentelemetry
Thanks to so many maintainers, I ran out of space on the telescope #maintainermonth #ossummit #opentelemetry
If you missed my talk at #DevoxxUK on how we gained #observability into our CI/CD pipelines, check out my post on the #CNCF blog:
https://www.cncf.io/blog/2023/05/05/how-to-gain-observability-into-your-ci-cd-pipeline/
#cicd #releasemanagement #devops #jenkins #opentelemetry #wtfissre
Engineer: What if we instrument our applications, and we don't need the data?
SRE: What if we don't, and then we do?
There is a marginal performance cost to instrumentation in applications using #OpenTelemetry. However, the #observability upside when you need it is far greater.
Hola 👋! Will you be attending #springio Barcelona? So will I 💪🏾! I'll be co-presenting with my good friend @TommyLudwig where we will talk about #observability #micrometerio #springboot #opentelemetry and more! Check our session here https://2023.springio.net/sessions/observability-of-your-application/ #java
This made my day @mipsytipsy. And also…TRACES FOR THE WIN!!
In case you missed the @oncallmemaybe episode with #opentelemetry superfan Iris Dyrmishi, y’all can check it out here ⬇️⬇️
https://oncallmemaybe.com/episodes/i-heart-observability-with-iris-dyrmishi-of-farfetch
OpenTelemetry Operator. Tracing made easy | by magsther https://medium.com/@magstherdev/opentelemetry-operator-d3d407354cbf
#Java #opentelemetry #Kubernetes
I had some more fun exploring manual #OpenTelemetry instrumentation for Langchain and LlamaIndex. It seems that writing an Instrumentor using wrapt's "wrap_function_wrapper" is the most sensible way for instrumenting libraries. Being stuck with callbacks or base class extensions is far from fun...
RT jcon_conference: See @\nicolas_frankel at #JCON2023 in Cologne talks about 'Practical Introduction to #OpenTelemetry Tracing'
Tracking a request’s flow across different components in distributed systems is…
Get your free #JUG Ticket: http://www.jcon.one
Had fun on the stream yesterday, we tried to get the #opentelemetry demo running in EKS using the ALB ingress controller using Terraform and Helm... it was... an experience...
We also delivered on the promised profanity levels.
What's this you say? You also want a UI for OpenTelemetry? Me TOO!!!
otel4s-0.2.1 is here with bugfixes, a rough draft of propagation, observable instruments, and more.
https://github.com/typelevel/otel4s/releases/tag/v0.2.1
We don't talk about 0.2.0.
So, #GCP doesn't trace by default all requests which leads to #Python #opentelemetry tracers to fallback to a non recording trace. This is problematic for scenarios where your client sends requests at high rate and these requests are very diverse. Your traces are basically missing all over the place. One approach is to force the Python tracer to trace even when parent traces were not set. But I wonder the impact on costs and performances.
I had no idea that #fluentbit had so many features! Especially support for #OpenTelemetry i will have to look into it again
just in time for #KubeCon: #elastic Common Schema (ECS) will join a single common schema format with the #OpenTelemetry Semantic Convention
naming conventions and mappings don't sound exciting but this moves you from a collection of fields to meaningful and actionable information
https://opentelemetry.io/blog/2023/ecs-otel-semconv-convergence/
How to enable #OpenTelemetry traces in #React applications | #RedHat
#javascript
@msroest You should look into #opentelemetry then look at the vendors that accept OTel data. We're one.
Observability is the evolution of what you'll have used before for understanding applications in production, and things have come a long way.
https://devblogs.microsoft.com/dotnet/announcing-dotnet-monitor-in-net-6/ - #dotNET #monitor: collect logs from a .NET process. I'm curious to see how this will evolve with #OpenTelemetry.
Filed under reasons why I love my job: getting to collab with the awesome folks at Syntasso, combining my love of #PlatformEngineering and #OTel! Check out my guest post on the Syntasso blog about how I used Kratix to deliver #Jaeger, the #OpenTelemetry operator pre-configured to send traces to Jaeger, and a little #golang service (to test the whole setup) on #kubernetes.
https://www.syntasso.io/post/guest-blog-lightstep-and-kratix
Shoutout to @abangser (who was an @oncallmemaybe S1 guest 😉) & Chris Hedley for all the help!
We're super excited to announce the release of #SpringCloud Sleuth #OpenTelemetry 1.1.3. Check the release notes for more information https://github.com/spring-projects-experimental/spring-cloud-sleuth-otel/releases/tag/v1.1.3 #Java #SpringFramework #SpringBoot
Don’t abuse log-correlation by @\kittylyst in his “#Observability for Java developers talk” @\devnexus
Live in 10 minutes at https://twitch.tv/martindotnet doing some Outside-in testing in #dotnet with a little #OpenTelemetry
Going live in an hour and I'm going to add some outside-in tests to the #OpenTelemetry #dotnet proxy. I'll also be adding some tracing to the app so it produces its own traces.
It should be interesting as we'll be working with HttpProbuf in the tests.
Inching closer to the upstream release of #Quarkus 3.0.0.Final!
In this latest Alpha release you can add #OpenTelemetry for JDBC with a simple setting, and the upgrade process from 2.x to 3 has been improved.
Read all about it: https://quarkus.io/blog/quarkus-3-0-0-alpha6-released/
#Java
On the stream tomorrow, I'll be adding an "outside-in" test harness for my #OpenTelemetry #dotnet proxy. This will form the basis of how I prove out that tail sampling is actually working as specified.
I may also have it create spans of its own too... during the test... so we can make sure its doing what we expect. That's if we get time.
Probably around 11am BST at https://twitch.tv/martindotnet
Going live NOW to see if I can get #dotnet to receive #OpenTelemetry data and maybe even forward it on as the start of the journey to understanding the impact of Tail-Based Sampling in Tracing
Join me... Hopefully it won't be as rough as yesterday!
Going live at 11:30am (30 minutes from now) to see if I can get #dotnet to receive #OpenTelemetry data and maybe even forward it on as the start of the journey to understanding the impact of Tail-Based Sampling in Tracing
Join me... Hopefully it won't be as rough as yesterday!
Going Live Now!!!
https://twitch.tv/MartinDotNet
Writing a Tail based sampler in #dotnet for #OpenTelemetry
Change of plan for the stream. I'm going live at 3pm BST on https://twitch.tv/MartinDotNet and I'll be creating a proxy for #OpenTelemetry in #dotnet as the start of an investigation into why Tail Based sampling is so hard.
I think I'm going to stream creating an #OpenTelemetry proxy in #dotnet later today if anyone is interested.
This is the start of me building some content around how Tail based processors work and using #dotnet to make it work.
Probably about 2 hours from now (12:30pm BST)
Tons of useful resources in @dnsmichi’s newsletter (and thanks for the shout-out for my book!)
---
RT @dnsmichi
@PolarSignalsIO @gitpod @ciliumproject @Odigos_App @AMD @awscloud @huggingface @GroupThink @raycastapp @awkwardferny @ClickHouseDB @MermaidChart @felipecruz @rawkode @kunalstwt @SaiyamPathak @LiliCosic @mhausenblas @horovits @NetflixEng 🔖 Book'mark
Practical #OpenTelemetry by @dan_gomezblanco
Resources for becoming a better pu…
https://twitter.com/dnsmichi/status/1637897639273418752
👁️ Observability
@lili wrote a fantastic piece about #Observability in practice
@mhausenblas announced their #opentelemetry in use podcast
@horovits dicussed FinOps Observability: Monitoring Kubernetes Cost with OpenCost
From Monitoring to Observability: eBPF Chaos recording available
Netflix time series metrics system (Atlas) and dashboard system (Lumen)
New opsindev.news #newsletter, feat. Polar Signals Let's Profile, learning #eBPF and AI/ML cont., @gitpod signing commits, #DockerHub changes, #Observability in practice, #OpenTelemetry in use, otel-desktop-viewer
https://opsindev.news/archive/2023-03-20/
Let's peek into the details 👇
The getting started docs for #OpenTelemetry and #dotnet have had a major update to make it a lot easier to follow and be in line with good practices.
https://opentelemetry.io/docs/instrumentation/net/getting-started/
Do let me know if there's something missing here or something you'd like to see!
#observability related question for anyone who happens to build or run a tracing backend: what happens if I send duplicate spans?
Super stoked to be speaking as part of a panel discussion with my #otel pals @cyberynn @reesesbytes @libbymeren @austinlparker at Open Source Summit North America 2023 🎉🎉
I published a new article on dev.to yesterday:
https://dev.to/ciscoemerge/trace-through-a-kafka-cluster-with-rust-and-opentelemetry-2jln
I was curious to find out how to connect Kafka producer and consumer spans in a #Rust application, using #opentelemetry
It turned out it's more straightforward as it might seem at first sight. In fact, it works almost exactly the same way as it does for HTTP. The only difference: I needed to implement two OTEL traits so I could use the standard OTEL context propagation mechanisms
The single best piece of advice I can give when building your #observability in your code with #OpenTelemetry is to use constants in the code for attribute names.
What you're doing is creating an index of where these are used. You're ensuring consistency so that you can query like things, and that your debug experience will be smoother.
Here's an example in #dotnet
I got to spend time learning #OpenTelemetry with #dotnet. I got one of our services at work writing to an OpenTelemetry collector which then forwards traces to App Insights. I’m hoping that sets us up for our move to Honeycomb. It’s nice to get to write some C# again.
Just adding a section on head sampling in #opentelemetry #dotnet (and tail sampling in the collector) to my talk in.... 4 hours time...
Why yes... I am an idiot... why do you ask?
I'm sorry, I've tried. I tried REALLY hard today to find the usecase, I wen through a load of old production apps, searched the interweb, I just couldn't find it
If you have a valid usecase for manually generating application metrics please tell me
Closest I got was Sales volumes, which are more business than debugging, and I'd probably suggest run it from the datastores if you can
All the other metrics are framework or "pseudo-infrastructure" metrics, not manual ones
Working on finishing up the demo for Practical #OpenTelemetry in #dotnet, and I'm using the tracing information to debug why things aren't doing what it should be doing.
Not opened a debugger at all, the debugging cycle is glorious...
Come in... the water's great.
Anyone done a lot with #OpenTelemetry on the web? We use it a bit but would really like to take it a lot further in our #nextjs app.
#OpenTelemetry helps us to get a proper observability and insights into our process and full tracing while investigating our system behaviour.
#EventSourcingallow us to enrich that with business data; we can combine tracing with business facts we store. Typically, we put them into event metadata. By that, we can see correlate technical traces with business information.
Read more 👇
https://event-driven.io/en/set_up_opentelemetry_wtih_event_sourcing_and_marten/
Feedback is welcome!
Managed to get a couple of examples of using #azure functions and #dotnet with #opentelemetry done today
There's some nuance that InProcess can only use 1.3.1 of OpenTelemetry and Isolated Functions needs a new middleware.
I also added the detection code for standard Azure attributes that come in through EnvVars. This is a lot better in 1.4.1 with resource detectors and the new Cinfigure methods.
Take a look
https://github.com/martinjt/otel-azure-functions
(Readme is coming)
I’ve enjoyed building otel-desktop-collector a lot, but now I’m looking for a full-time gig.
I’ve done embedded systems, games, android, and web dev, but I’m really into Go and #OpenTelemetry these days and would ideally love to find work in this field.
DMs are open!
Hi! So, I made a thing…
And by “thing” I mean a CLI tool to help developers work locally with #OpenTelemetry and visualize their traces. I hope y’all like it better than reading spans on stdout. Also, it has an axolotl.
Been playing with getting some fancy dashboards and whatnot going on, with a bunch of metrics, traces and logs, to see how to make a bunch of #DotNet services observable.
Looking good! 😁
Hit some challenges, but for the most part, got things working as I hoped. Now that it's coming together, need to clean things up a bit and write a blog post (or more) on the subject 😅.
Featuring #DotNet, #Kafka, #PostgreSQL, #Grafana, #Tempo, #Loki, #OpenTelemetry
Having a bad time trying to get axum-tracing-opentelemetry to talk to Jaeger and I’m sure that I’m just not holding it right or whatever, but I’m going to step away for a bit. Also, probably need to do some housework.
#rust #rustlang #axum #opentelemetry
Asking for a friend. Is anyone hiring for golang work in Canada? 👀 Especially anything #OpenTelemetry adjacent?
Boosts appreciated
Had a few false starts integrating #Elixir with #Honeycomb using #OpenTelemetry, so when we got it working, I decided to write down what worked for us: