#S3

Usually I just share my #art here, but today I was busy helping a friend set up their #website.

So instead, here's my recipe to set up #StaticWebsites using #Ansible to manage #AWS #S3.

https://edward.delaporte.us/blog/s3/

🚀New blog post! Learn how to efficiently move your S3 files with the AWS CLI. #AWS #S3 📁🔄
https://lpembleton.rbind.io/posts/aws-cli-s3/

I have a very brief favour to ask: could anyone check how the #AWS #S3 #API responds when one tries to delete an object that simply doesn't exist, in an object-versioning-*disabled* bukkit?

Do you get an HTTP 204 ... or something else? #lazyweb

(I do know how lazy this sounds - but I'm in the middle of a hike (without a decent computer!) and have had a fab idea that hinges on knowing about the API response!)

#S3, #S4: Störung beendet (Stand 30.09., 08:57 Uhr)

30.09.2023, 08:25 Uhr - 10:00
Es kann noch zu Verspätungen und Teilausfällen kommen aufgrund eines vorangegangenen Polizeieinsatzes in Langen.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143309%26view=trafficmap%26trafficmap_date=30.09.23#map

#S3, #S4: aktuelle Störung (Stand 30.09., 08:28 Uhr)

30.09.2023, 08:25 Uhr - Dauer unbekannt
Es kommt zu Verspätungen und Teilausfällen aufgrund eines Polizeieinsatzes in Langen.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143309%26view=trafficmap%26trafficmap_date=30.09.23#map

#S1, #S2, #S3, #S4, #S5, #S6, #S7, #S8, #S9: aktuelle Störung (Stand 30.09., 03:35 Uhr)

30.09.2023 02:54 - Dauer unbekannt
Wegen kurzfristigem Personalausfall kommt es im gesamten S-Bahn Netz zu vereinzelten Zugausfällen.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143306%26view=trafficmap%26trafficmap_date=30.09.23#map

#S1, #S2, #S3, #S4, #S5, #S6, #S7, #S8, #S9: aktuelle Störung (Stand 30.09., 03:35 Uhr)

30.09.2023 02:54 - Dauer unbekannt
Wegen kurzfristigem Personalausfall kommt es im gesamten S-Bahn Netz zu vereinzelten Zugausfällen.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143306%26view=trafficmap%26trafficmap_date=30.09.23#map

Version 4.4.0 of syslog-ng is now available in #FreeBSD ports:

https://www.freshports.org/sysutils/syslog-ng/

Among others it adds support for #S3 as a destination. You have to enable #Python modules for this.

#LogManagement

RT #S3 fährt aktuell nicht zwischen Buxtehude und Stade. Grund: Technische Störung an der Strecke. Ersatzverkehr wird eingerichtet. #hvv #sabhnhh

On my way to #manchester to join AWScomsum! 🛫

I look forward to meet amazing people there, learn from the speakers and share some learnings about #S3 pre-signed URLs!

If you are there too come over for a chat 🤝

https://www.jofogas.hu/pest/Uj_Samsung_Gear_S3_szilikonszijak_tobb_szinben_141841112.htm

#jófogás #ungary #samsung #gear #s3 #frontier

#S3, #S4, #S5, #S6: Einschränkungen in den Nächten vom 26.09. bis 30.09.

26.09.2023, 20:00 Uhr - 30.09., 06:00 Uhr
Es kommt aufgrund von einem unterbesetzten Stellwerk in "Frankfurt Südbahnhof" zu Einschränkungen.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143237%26view=trafficmap%26trafficmap_date=26.09.23#map

My unofficial syslog-ng 4.4.0 packages for @opensuse / #SLES are now available:

https://www.syslog-ng.com/community/b/blog/posts/installing-latest-syslog-ng-on-rhel-and-other-rpm-distributions

New features include:

- #Grafana #Loki and #S3 destinations (only in Tumbleweed)
- http() compression

Support for #Riemann temporarily disabled due to compile problems.

My unofficial syslog-ng 4.4.0 packages for @fedora / #RHEL are now available:

https://www.syslog-ng.com/community/b/blog/posts/installing-latest-syslog-ng-on-rhel-and-other-rpm-distributions

New features include:
- #Grafana #loki destination (only on Fedora)
- #S3 destination (not on RHEL 7)
- http() compression
And more.

Version 4.4.0 of syslog-ng is now available. New features include:

- #Grafana #Loki destination
- #S3 destination (implemented in #Python)
- compression in the http() destination
- @OpenSearchProject destination

For more details read the release notes at: https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-4.4.0

#LogManagement

#S1, #S2, #S3, #S4, #S5, #S6, #S8, #S9: aktuelle Störung (Stand 26.09., 06:56 Uhr)

26.09.2023 05:56 - Dauer unbekannt
Es kommt zu Verspätungen und Ausfällen auf allen S-Bahn-Linien außer der S7. Die Linien S5 und S6 fahren im 30-Minuten-Takt.Grund dafür ist eine Signalstörung im Frankfurter S-Bahn-Tunnel.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143219%26view=trafficmap%26trafficmap_date=26.09.23#map

Hello MastoAdmins, I need help! Buddyverse.xyz is currently serving files from link.storjshare.io. However, I want to use my own domain, "cdn.buddyverse.xyz," now. Is there any potential issue if I change S3_ALIAS_HOST at this point? The old files should be accessible from their previous URL. Do I need to perform any migrations or similar tasks? Thank you! 😊

Edit: I simply changed S3_ALIAS_HOST and everything is working fine so far

#MastoAdmin #Fedimin #FediAdmin #Mastodon #Fediverse #Askfedi #Help #Sysadmin #S3

Kennt sich jemand mit dem ObjectStorage von IONOS aus? Was kommen da für Traffic-Kosten, bei einer durchschnittlichen Mastodon Instanz, auf einen zu? #mastoAdmin #admins #s3 :boost_ok:

RT #S3 Züge mit Start und Fahrtziel Buxtehude fahren abweichend bis Neugraben. Mit Fahrtziel Stade fahren planmäßig. Grund: Signalstörung. Es kann zu Verspätungen kommen. #hvv #sbahnhh

RT Update S3: Technische Störung zw. Pinneberg u. Elbgaustraße wurde behoben, die #S3 fährt wieder wieder von und nach Pinneberg. Der Ersatzverkehr wird eingestellt. #hvv #sbahnhh

RT S3: #S3 fährt aktuell nur zw. Stade - Hbf - Elbgaustraße. Grund: Technische Störung an der Strecke. Zw. Elbgaustraße u. Pinneberg den Ersatzverkehr mit Taxis benutzen. #hvv #sbahnhh

RT #S1, #S3 und #S31 werden aktuell zw. Altona u. Hbf über Holstenstraße u. Dammtor umgeleitet. Die Halte zw. Königstraße u. Jungfernstieg entfallen. Grund: Feuerwehreinsatz. Bitte auf Busse ausweichen. #hvv #sbahnhh

I tried firing up a #Windows instance in #AWS #EC2. Super-easy, fast, and doesn't cost anything when not in use (an $0.12/hr when in use). You can snapshot the #EBS volume to #S3 and delete/restore it later to save more. If you’re like me and only need access to Windows periodically, it's a great way to go.

#S3, #S4, #S5: Störung beendet (Stand 22.09., 09:038 Uhr)

22.09.2023 10:30
Es kann noch zu Folgeverspätungen kommen aufgrund eines vorangegangenen Polizeieinsatzes zwischen Frankfurt-Westbahnhof und Rödelheim.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143134%26view=trafficmap%26trafficmap_date=22.09.23#map

Hey #mastoadmin s , need your help.

After I upgrade my server to 4.2.0, and when I try to set

- S3_STORAGE_CLASS=ONEZONE_IA

I constantly start to get

Aws::S3::Errors::InvalidStorageClass

errors.

This is same with the STANDARD as value as well.

Using Scaleway as a backend, and I know they have this feature on my region (FR-PAR). https://www.scaleway.com/en/docs/storage/object/api-cli/object-operations/#putobject

Have you experienced this after the update? If so, can you help me please?

Thanks!

#aws #s3 #storage #onezone #onezone_ia #scaleway

Boost willkommen :neocat_sign_thx:
Kleine Info noch:

Die App S3Drive gibt's jetzt auch im Flathub :neocat_sign_yip:

https://flathub.org/apps/io.kapsa.drive

Desweiteren steht ein Appimage und DEB Paket per GitHub bereit.

https://github.com/s3drive/app

Im Microsoft Store ist es schon länger vertreten, eine Exe findet sich aber auch bei GitHub, genau wie die Mac Version.

Generelle Informationen auf der Homepage

https://s3drive.app

#s3 #s3drive #flathub #flatpak #linux

RT Die Züge der #S3 mit dem Start/Ziel Buxtehude beginnen/enden abweichend in Neugraben. Grund: Technische Störung an der Strecke. #hvv #sbahnhh

#S1, #S2, #S3, #S4, #S5, #S6, #S8, #S9: aktuelle Störung (Stand 19.09., 19:26 Uhr)

19.09.2023, 19:25 Uhr - Dauer unbekannt
Es kommt zu Verspätungen und Ausfällen. Grund dafür sind zum einen betriebsfremde Personen im Gleis und zum anderen zwei Rettungseinsätze in Rödelheim und in Bad Soden.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143055%26view=trafficmap%26trafficmap_date=19.09.23#map

#S3: aktuelle Störung (Stand 19.09., 09:00 Uhr)

19.09.2023, 08:45 Uhr - Dauer unbekannt
Es kommt zu Verspätungen aufgrund einer Störung an der Strecke bei Darmstadt Hauptbahnhof.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143043%26view=trafficmap%26trafficmap_date=19.09.23#map

J'ai corrigé un problème sur l'instance Mastodon de #FédiQuébec ces dernières heures.
Des fichiers médias associés à des messages datant d'environ un mois et s'étalant sur une période d'environ une semaine de certains comptes distants étaient manquants. C'est maintenant corrigé.
26 Go de plus dans le bucket #S3.

#S3, #S4, #S5, #S6: Störung beendet (Stand 17.09., 09:21 Uhr)

17.09.2023 10:00
Es kann noch zu Folgeverspätungen kommen aufgrund einer vorangegangenen Störung wegen eines Gegenstandes im Gleis im Bereich Frankfurt-Lokalbahnhof.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143001%26view=trafficmap%26trafficmap_date=17.09.23#map

#S3, #S4, #S5, #S6: aktuelle Störung (Stand 17.09., 09:16 Uhr)

17.09.2023 - Dauer unbekannt
Es kommt zu Verspätungen und kann zu Teilausfällen kommen aufgrund eines Gegenstandes im Gleisbereich im Bereich Frankfurt-Lokalbahnhof.
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2143001%26view=trafficmap%26trafficmap_date=17.09.23#map

Pour #FédiQuébec, j'ai actuellement trois VPS "starter". Un pour le serveur courriel et le site web, un pour le serveur d'authentification et un pour Mastodon.
Si les deux premiers suffisent à leurs tâches, Mastodon manque d'espace disque malgré l'utilisation d'un #S3. Ça devient même compliqué de faire les mises à jour faute d'espace pour builder les nouvelles images docker.
Il manque aussi de RAM pour pouvoir utiliser ElasticSearch.
Bref, je songe louer un 4e VPS un peu plus gros...

#S3, #S4, #S5, #S6: Störung beendet (Stand 16.09., 19:52 Uhr)

16.09.2023 - Dauer unbekannt
Es kommt zu Folgeverspätungen aufgrund eines vorangegangenen Polizeieinsatzes in "Frankfurt Lokalbahnhof",
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2142996%26view=trafficmap%26trafficmap_date=16.09.23#map

#S3, #S4, #S5, #S6: aktuelle Störung (Stand 16.09., 19:20 Uhr)

16.09.2023 - Dauer unbekannt
Es kommt zu Verspätungen und ggf. kurzfristigen Teilausfällen aufgrund eines Polizeieinsatzes in "Frankfurt Lokalbahnhof"
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2142996%26view=trafficmap%26trafficmap_date=16.09.23#map

#S1, #S2, #S3, #S4, #S5, #S6, #S8, #S9: Störung beendet (Stand 16.09., 16:26 Uhr)

16.09.2023 - Dauer unbekannt
Es kommt zu Folgeverspätungen und ggf. kurzfristigen Teilausfällen aufgrund von Personen im Gleis in "Frankfurt Konstablerwache"
http://www.rmv.de/auskunft/bin/jp/query.exe/dn?selectedhimid=2142988%26view=trafficmap%26trafficmap_date=16.09.23#map

Anyone who tried out #fybe ? Its a brand owned by #contabo with relatively good #s3 / #objectstorage pricing.

I want to give it a shot because #b2 is quite costly on API calls, but I have been stuck for multiple hours in “provisioning”

Controversial topic time at AWS Bites! 😲 S3 is not a filesystem, but AWS made a new way for you to pretend that it is. Good idea 💡 or footgun-enabler 🙈?! Let's find out! https://www.youtube.com/watch?v=ArM0XaAwkrY #AWS #S3 #Buckets #filesharing #cloudmigration

As promised, we are back with a new episode of #AWS Bites #podcast!

Today we talk about mounting #S3 buckets directly in your filesystem using AWS Mountpoint, an interesting open-source project written in #Rust.

Is it a good idea though? 🤨

https://awsbites.com/95-mounting-s3-as-a-filesystem/

RT #S3 fährt aktuell nicht zw. Pinneberg und Elbgaustraße. Grund: Polizeieinsatz. #hvv #sbahnhh

RT Streckensperrung zw. Buxtehude <> Stade. Grund: Stellwerksstörung. #S3 #hvv #sbahnhh

RT #S21 und #S3 fahren aktuell nicht zwischen Altona und Elbgaustraße/Pinneberg. Grund: Polizeieinsatz. #hvv #sbahnhh

Have some #groovy running in an #ECS container on #Fargate. The #AWS #S3 client builder should be using the default provider chain but does not appear to be credentialing correctly. A subsequent object upload fails with a 403 Forbidden. The ECS task and execution role otherwise has the correct #IAM permissions and I'm able to upload a file directly from within the container.

Has anyone here used #minio to host a static website similar to how you can with #S3? #devops #sysadmin

There's some standard #AWS #security guidance out there that is (in my personal opinion) incredibly stupid. It is usually reported as: “Ensure MFA Delete is enabled on S3 buckets”. And you can find references to it in AWS CIS Benchmarks 1.4 level 1, under the heading “2.1.3 Ensure MFA Delete is enable on S3 buckets”.

Raise your hand if you do this. Put your hand down. You do not do this. No one should do this ever. Let me summarise the documentation on this feature. Here are the steps:

1. Login as root. (Error! We NEVER EVER DO THAT)
2. Provision Access Key / Secret Key to root and copy it to your favourite command line, like a terminal on your laptop or an EC2 instance. (Holy shit! We NEVER EVER DO THAT)
3. Go get the root user’s physical or virtual second factor (You're just fucking with me now.)
4. Issue the s3api put-bucket-versioning API call from your root AWS CLI with root’s access keys and root’s MFA token (💀​)
5. Undo all those things: revoke the access keys, deprovision the virtual MFA token from the admin’s phone, whatever.
6. If you have any reasonable AWS security in place monitoring root account usage, you now go close out a dozen false alarm alerts going off because the root account was used for something. Enjoy the rest of your day closing tickets.

And you have to do this for every bucket!? Every time someone creates a new bucket? This advice is like 7 years old. It should never have been mindless and recommended to all buckets.

The goal of this was about making sure that adversaries couldn't delete record of malicious activity or delete data they shouldn't. Since CloudTrail was often stored in S3, you needed to protect your S3 objects from unauthorised deletion. The capability that you really want is called S3 Object Lock. It doesn’t require root access keys and root’s MFA and the CLI. It stops object from being deleted, which is what this is all about. It has a few modes of operation, with lifecycle policies, and reasonable behaviours.

I can’t believe that the CIS Benchmarks still call for MFA delete. To me it goes to show that they don't actually know how the platform works. The AWS conformance pack for S3 no longer has MFA delete in it. They still mindlessly recommend S3 versioning (which I think is only useful in some rare use cases, but whatever), but they don’t recommend MFA delete any longer.

I hope people aren’t out there doing #S3 #MFA delete in 2023. It is stupid.

Let's say I inspired you and you decided to turn it off where you might have turned it on. Good luck with that. (a) you can't see it in the AWS console (you can only get the status from the AWS CLI), and (b) you gotta do the same dance above (root's keys and MFA) to turn it off.

RT Streckensperrung zw. Pinneberg <> Hamburg Elbgaustraße. Grund: Polizeieinsatz. #S3 #hvv #sbahnhh

File storage in #AWS is simple: just use Simple Storage Service (#S3)

...unless you are using Elastic Cloud Compute (#EC2); in which case you can use instance volumes

...unless you want the files to persist when EC2 restarts; in which case you must use Elastic Block Store (#EBS)

...unless your EC2 instances are in different availability zones; in which case you need Elastic File System (#EFS)

SEE!? SIMPLE! 😕

Adding S3-compatible cloud storage to Pixelfed

Pixelfed is a federated photo sharing service that is an alternative to Instagram and uses ActivityPub to share posts across other services such as Mastodon, Pleroma and indeed WordPress. It is one of the more mature Fediverse apps and has been in continuous development for s

https://simongreenwood.me.uk/adding-s3-compatible-cloud-storage-to-pixelfed/

#CloudStorage #Fediverse #Pixelfed #Technical #Uncategorized #fediverse #pixelfed #s3

Greetings, Fediverse. Introducing BonitoCache, optimized for S3 Buckets reads. Crafted in Go, it allows configurable RAM and disk limits. Preliminary data shows a 95% bandwidth reduction to the S3 endpoint.

Optional InfluxDB integration is available for performance monitoring, providing in-depth metrics.

Your feedback is invaluable. For further information, visit our GitHub repository: https://github.com/i5heu/bonito-cache

#foss #s3 #opensource

C'est fait. À première vue, c'est beaucoup plus rapide.
#ObjectStorage #S3 #OVH

J'suis pas vraiment satisfait du #S3 Standard d'#OVH. C'est plutôt lent. Je songe déplacer le contenu de mon bucket pour Mastodon vers un bucket S3 High Performance.

Oha! Die S-Bahn wird auf der Strecke geräumt. 😲

Alle sBahnen über Hauptbahnhof fallen aus. 😕

#SBahn #S1 #S11 #S3 #S31 #Hamburg

RT #S1 fährt nicht zw. Hasselbrook <> Altona. #S21 fährt nicht zw. Tiefstack <> Sternschanze / Altona. #S3 fährt nicht zw. Altona <> Hammerbrook. Grund: Personenunfall am Hbf. #hvv #sbahnhh

Managed to migrate from one @portainerio host to the other using the #S3 backup feature via #Synology #ObjectStorage.

The instance had 4 more agent-connected hosts, that were all registered with 0 issue and all compose deployments are working!

https://www.blackvoid.club/backup-portainer-with-synology-c2-object-storage/

🤯 Je galère comme pas permis pour uploader un fichier depuis mon programme #nodejs / #typescript vers mon bucket #minio / #s3

Ça me saoule...

Need a tool to manipulate objects hosted on #S3 buckets?
There is #s5cmd, it's performance are great!

https://github.com/peak/s5cmd

If anyone knows an equivalent in #rust, I'm interested

Ich würde sagen, der #S3 #Objektspeicher läuft jetzt auch auf #Mastodon 😀

Auf #Firefish sieht es gut aus mit dem #S3 #Objektspeicher. Jetzt synchronisiere ich die sich bisher auf #Mastodon angesammelten Dateien, bei gestoppter Instanz.

Da ich den Cache heute auf einen Tag begrenzt habe, sind es auch nur 23 GB 😉

Wenn dies beendet ist, starte ich Mastodon mit der geänderten Konfigurationsdatei neu, und hoffe, dass alles glattgelaufen ist 😀

Um mir keine Gedanken über den #Speicherplatz auf meinem #Server machen zu müssen, teste ich gerade den #Scaleway #S3 Anbieter und habe meine #Firefish Instanz bereits umgestellt. Bisher sieht es gut aus.

Wenn es fehlerfrei funktioniert, stelle ich auch meine #Mastodon Instanz um 😀

Decided to supplement my local #S3 storage (min.io) with some off-site S3 storage. Bought 1 TB of S3 #Cloud storage from #iDrive for $20/year. Seems like a good deal. Using the s3cmd to sync local directories to iDrive but there are many GUI apps that support S3 as a destination.

Holy crap. Backblaze this is awesome!!

#backblaze #s3 #s2 #cdn #mastoadmin

Look out #ILGISA, I did it again. I submitted 4 things to present. And ILGISA just keeps saying yes! One of us will learn at some point.

For you IL #GIS folks, if you are interested in

- Using #Survey123 for restaurant inspections

-Using the #OSMUS Tasking Manager to oversee #OpenStreetMap editing campaigns

-Creating local extracts of OSM data with #osm2pgsql

-Using Amazon #S3 for a "serverless" basemap tile option

Then you should come to ILGISA this year!

☁️ A Look at Using Elixir Streams, Elasticsearch & AWS S3
➥ Guy Argo | HackerNoon

https://hackernoon.com/a-look-at-using-elixir-streams-elasticsearch-and-aws-s3

#Elixir #MyElixirStatus #AWS #S3 #Elasticsearch

It’s been a pretty successful week so far. Got a #lambda function to update our #S3 bucket policies and #sql backups direct to an S3 bucket accomplished too. Off to a good start!

#AWS and the #cloud is a fun place to be right now.

Advice please?

I'm trying to get my head around #S3 storage costs. I think I pay for the amount of data in storage per month, AND data transfer costs. As an example, Amazon allow free data transfer IN but charge per GB out.

How do you estimate what the transfer costs could be?!

Any tips/thoughts/good providers? I just used Amazon as an example.

[ Mountpoint for Amazon S3 Now GA to Access Bucket Like Local File System ]
https://www.infoq.com/news/2023/08/mountpoint-amazon-s3-ga/ #AWS #S3 #MountPoint

「 #AWS 、 #Linux から #Amazon #S3 をファイルシステムとしてマウントし利用できる「Mountpoint for Amazon S3」正式公開」： Publickey

「Amazon S3のバケットをまるでローカルのファイルシステムのように扱えるようになります。」

https://www.publickey1.jp/blog/23/awslinuxamazon_s3mountpoint_for_amazon_s3.html

#prattohome #Publickey

🗞 New episode of Changelog News!

🤔 Armon Dadgar announces #HashiCorp's #BSL future
🏆 Matt Rickard on why #TailwindCSS won
🕴️ WarpStream is like #Kafka directly on top of #S3
🧩 Vadim Kravcenko’s guide to managing difficult devs
📢 Russ Cox gives an update on #golang 2
🎙 hosted by @jerod

🎧 https://changelog.com/news/57

Has anyone of you done a performance comparison between Restic with SFTP backend and S3 backend?

My _guess_ would be that S3 is significantly faster.... :thaenkin:

#s3 #restic #backup #sftp

#aws #s3 uses #rust to define its critical components https://www.amazon.science/publications/using-lightweight-formal-methods-to-validate-a-key-value-storage-node-in-amazon-s3

We are working on adding support for authentication with #S3 backends using temporary credentials from #STS obtained from an ID token from an #OpenID connect provider. Let us know if you know about a common deployment scenario (on #AWS or other provider). We are looking to test the interoperability of our implementation and provide sample connection profiles in https://github.com/iterate-ch/profiles/issues/55

There was a loud 'WHAT THE ACTUAL FUCK!?" to be heard in this house after Carrie Paige screamed her terrifying scream and it was lights out in the Palmer house.

I feel so sorry for first timers and what they have to go through watching it. Fuck you, Lynch 🥺.

#twinpeaks #s3

How about that ominous whooshing guys and girls. #twinpeaks #s3

整理了一张个人认为比较适合Fedi建站的平价S3对象存储提供商价格比较表，希望有帮助。

#长毛象运维 #长毛象建站 #S3 #对象存储 #登基 #长毛象站长联谊会

detailed and genuinely useful blog on how we built our internal data warehouse on #ClickHouse at ClickHouse

https://clickhouse.com/blog/building-a-data-warehouse-with-clickhouse

#ClickHouse #DataWarehouse
starring
#Superset #AppFlow #Airflow #Redis #Docker #opensource #RDS #PostgreSQL #DBT
and including data from
#Salesforce #AWS #S3 #BigQuery #M3ter #Segment #Marketo

Aktuell #S21 als #S2 über Jungfernstieg und Landungsbrücken. #S31 wird nun eingestellt. Evakuierung abgeschlossen. Die Linien #S1, #S3 ebenfalls mit vielen Abweichungen und großen Verzögerungen. Grund: Fliegerbombe in Sternschanze #hvv #sbahnhh

Je suis à la recherche d'un #MastoAdmin ou des #MastoAdmins qui utilise-nt un service #ObjectStorage #S3 et qui voudrai-en-t bien me donner un p'tit coup de main pour mettre ça en place.

En 2018, j'avais suivi ce guide (https://stanislas.blog/2018/05/moving-mastodon-media-files-to-wasabi-object-storage/) pour utiliser #Wasabi sans #ReverseProxy, mais comme ce n'est plus possible (voir: https://pouet.fedi.quebec/@manu/110658367412016709) j'essaie avec #OVH (S3 Standard) et un reverse proxy inspiré de ça : https://docs.joinmastodon.org/admin/optional/object-storage-proxy/

Merci de partager !

"Hijacking S3 Buckets: New Attack Technique Exploited in the Wild by Supply Chain Attackers"

https://checkmarx.com/blog/hijacking-s3-buckets-new-attack-technique-exploited-in-the-wild-by-supply-chain-attackers/

"Without altering a single line of code, attackers poisoned the NPM package "bignum" by hijacking the S3 bucket serving binaries necessary for its function and replacing them with malicious ones"

#security #supplychain #npm #javascript #s3 #aws

#Wasabi, offrant un service #S3 (Object Storage), n'offre plus le egress publique aux petits clients (moins de 25 To), forçant donc le recours à un reverse proxy pour servir les fichiers stockés dans le bucket.

Connaissez-vous des alternatives?

@palmin Wow, this works perfectly! I was connected to my B2 buckets in just minutes. Instant buy.

#backblaze #b2 #aws #s3 #ios

@Kaea
Jeśli chcesz ograniczać koszty, inwestujesz we własną infrastrukturę, na której oferujesz usługi. Bo zabawa z cloud od korpo zawsze wychodzi drogo :)

FYI: Fundacja @ftdl posiada własną mini serwerownię w Krakowie, zasilaną w dużej mierze z fotowoltaiki. Oraz własne zespoły admińskie, moderacyjne, pomocy i R&D.
Prowadzi instancje #Mastodon, #Matrix, #Nextcloud, #Kbin, #S3 oraz wspomaga projekty infrastrukturą jak Noevil.pl, Kooperatywy Parasol i Open Food Network, SVMetaSearch. Admini wspierają również największą instancję /kbin.social
Wszystko to bez abonamentów $, reklam czy państwowych dotacji.

To inna skala, ale gdy jest wola, to się da :)
@manganapp

Cool! @palmin just published a #S3 #ObjectStorage app for #iOS and #MacOS.
After his amazing #git client https://workingcopy.app/ and his #SSH terminal/files-by-SSH app https://secureshellfish.app/ I am sure we can expect a great app. It‘s called „S3 Files“ and you can read more at https://s3files.app/

If you've been following my recent #MinIO troubles: there's been a new development!

They have...

...decided to drop any hint of cooperation and lock down the issue.

And you know what: I'm feeling good about this. I can't drop MinIO for work, but personally, I won't touch the project with a ten-foot pole.

But here's a final reminder: if you're using encryption, make sure that you're running the latest version of MinIO's KES server. Otherwise your data could be corrupted.

#S3 #ObjectStorage

From a #matrix room, #AWS #S3 egress costs are like a #ransomware 😂

Heads-up, #NixOS Foundation seems to need help and community input on a developing NixOS cache S3 situation: https://discourse.nixos.org/t/the-nixos-foundations-call-to-action-s3-costs-require-community-support/28672

Please don't hesitate to bring new points and interesting things which could help steering the situation!

I already gave my (somewhat personal) view in the second post.

#NixOS #S3 #Cache

#aws just emailed to say some client is accessing stuff in #s3 using old tls, and if i want more information i should set up something called #cloudTrail. i'm twenty minutes into the docs and console and i've decided i no longer care about the client using old tls.

Of course, storage costs only. Managing cost: 0.0025 per 1000 objects. I have ~10k objects. So cost: $0.025 for S3 to figure out what to move and when 😂 Just pushed another ~110gb of pictures. (last lines). Intelligent tiering is awesome

#s3 #aws

HIRING: Senior Principal Engineer, Encryption / Sydney, Australia https://infosec-jobs.com/J30998/ #InfoSec #InfoSecJobs #Cybersecurity #jobsearch #hiringnow #CyberCareers #Sydney #Australia #APIs #AWS #Azure #Cloud #Encryption #GCP #Java #Lambda #Microservices #S3 #Strategy

I wrote a blog post about my #Mastodon #S3 migration from #Scaleway to my own #Minio media storage:

Switching Mastodon from Scaleway S3 to self-hosted Minio S3 media storage
https://thomas-leister.de/en/switching-mastodon-from-scaleway-to-selfhosted-minio-s3/

Maybe it is an interesting read for some of you. 🙂

@mattblaze
Wasn't there a person who was the whole #AWS #S3 on #Bluesky? I'm not sure their verification method is more secure at all.

T-Shirts for #ICSE2023! I am hiring multiple Post-Docs in software testing and analysis who want to combine their independent research agenda with our #S3 techniques. Ask wearers (+ me) at #ICSE for details, and check out the S3 ERC project at https://cispa.de/s3

@marcthiele @normanposselt Same here … prepared for selling #VanMoof #S3

I think I'll be able to close the Scaleway S3 bucket soon. Saves us another 9 € / month.

#s3 #metalheadclub

@charlotte @erk @encthenet EXACTLY!

#Amazon has a vested interest to act a bit more longterm.
Unlike #Microsoft's #EEE https://en.wikipedia.org/wiki/Embrace%2C_extend%2C_and_extinguish#Examples_by_Microsoft ] they want #S3 to become the de-facto standard, as they already dominate #CloudComputing and making shit easier on their platform will only work if it isn't exclusive.

Even if that means Microsoft ( #Azure ), #OVH, #Hetzner and even #Proxmox can do the same...

It also fixes a lot of issues #iSCSI has...

@erk

@kkarhan

Also, requiring people/companies to pay doubly ensures that not everyone's voice is heard. It's already time consuming enough, but having to pay thousands of dollars for the honor of spending your free time to try to fight against corporations isn't something most OSS people want to do.

There's a reason most RFCs are corporate sponsored.

#API #s3

@erk

@kkarhan

Except that there are other forums that produce standards that are open/free. I've participated in an OASIS standard. In checking to see if ecma made all their standards free, I came across this:
https://www.nist.gov/standardsgov/standards-organizations-offer-free-access-their-standards

Quite a few orgs making their standards free (though a number of the ones on that list have limitations).

#API #s3

Bilan garage après 6 mois d’utilisation :
* ça marche au global vachement bien ;
* en mode simple instance, il faut vraiment avoir du stockage ultra fiable derrière si on ne veut pas de mauvaise surprise ;
* en mode multi-instance, c’est hyper résistant même si le stockage/réseau ne sont pas particulièrement fiable derrière.

#rust #garage #s3

it would appear #aws #s3 bucket changes are rolling out in us-east-2 right now

https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/

#AWSCognito and #AWSS3 allow for subdomain hijacking / squatting. Please doublecheck spelling when communicating with #cognito and #s3 subdomains. #cybersecurity

Playing around with #NocoDB this morning and I noticed that it has an "app store" for plugins and other sorts of integrations. Chuffed to see #Scaleway in there by default to store attachments on the #S3 backend. I wonder what other things we could hook up there. :blobfoxthink:

Whoever named it #Route53 when #S3 already existed is a douchebag. #AWS

@cldellow So, one issue is, the way #AWS exports the snapshot to #S3, it splits up large DBs into multiple gzipped parquet files (See screenshot). I'm not sure if there's a joining process that needs to happen first, or if the tool is smart enough to just do it.

I can almost certainly send you a file to show you what’s what; dm me an email?

Thank god I have over a decade of #Python experience, so I have the skills necessary to get a basic working env.

Anyway, pipenv was the hammer this time, so now I have datasette installed, hooray!

Next: getting all the parquet files out of #S3 and onto my local machine.

Here, another hiccup! `aws s3 cp s3://{bucket}/{key}` returned “fatal error: An error occurred (404) when calling the HeadObject operation”

I must be doing _something_ wrong, because @panic's Transmit got 'em just fine.

3/n

Probably there have been a couple of these polls around, still I would like to know:
What object storage are you using for your #Mastodon #Instance (if any)? What are the advantages and disadvantages?
I am currently using #AWS #S3 which works great, but is probably not the cheapest option. Boosts appreciated. :boost_ok:

#MastoAdmin #Poll #ObjectStorage

Bon voilà, la méthode exhaustive, pas du tout scriptée parce flemme pour migrer des données d’un stockage local à un stockage S3 pour Nextcloud.

https://blog.libertus.eu/migrer-nextcloud-dun-stockage-local-a-s3/

En fait, par rapport à la version qu’on trouve le plus souvent sur le net, il manque essentiellement ce qui se passe dans la table `oc_mounts`.

J’aimerais tester pour voir si c’est possible de migrer compte par compte (à vue de nez, c’est impossible mais on ne sait jamais…).

#nextcloud #s3 #garage

@ghaering @jeffbarr
I could read the 2,700 words for 28 manual steps, making sure I select the right region for each of my >120 buckets in >40 AWS accounts with >5 million objects--and I'm just one individual customer.

Or AWS could automate it for all customers, making sure that the experience is smooth and transparent. It wouldn't even matter if it took a year to run.

If encrypting all future S3 objects is appropriate, then doing the same for existing objects is a clear choice.

#aws #s3

@jeffbarr
Next up: Encrypting all existing S3 objects transparently in the background?

#awsWishlist #aws #s3 #security #compliance

@danie10@mastodon.social

I'm not sure what your asking. You mean a description of the product?

It's distributed S3 compatible object store, self hosted for a redundant solution.

For example, for a #Fediverse information store, say, a #PeerTube live and #VoD #streaming server, you can use #Wasabi and purchase affordable storage space to stream your videos from on their S3 compatible buckets, or you can purchase inexpensive Storage VPSes and deploy your own distributed #S3 storage solutions using #Garage for much less than paying for additional HDD storage space on your own compute instances.

All of your videos can be kept and streamed from your S3 storage space much more affordably than paying for larger HDDs on your #VPS or it costing you an arm and a leg with Amazon (ewww, yucky).

Garage provides the distributed high availability and is more affordable than even discount commercial #S3 storage providers like Wasabi.

https://garagehq.deuxfleurs.fr/

There's additional use cases on the project's website, which is more descriptive than their download page for the latest release at their git repo.

Purchasing additional VPSes with fewer computing resources that are really cheap and optimized as storage servers as opposed to adding additional HDD space, or purchasing extra volumes, on your existing VPS is much cheaper, and Garage provides that distributed high availability. You can create buckets and your other VPSes can also take advantage of that additional space, instead of, for example, creating NFS volumes on one (by comparison, expensive) server for your other VPSes to mount.

You can further distribute that storage space globally with #Garage, which is self-hosted S3 compatible and #FOSS.

So you have 4 VPSes with lots of RAM and CPUs in each one, but they have small HDDs too keep costs down. You can easily outgrow your storage needs with fresh #Friendica (https://Friendi.ca), #Calckey(https://codeberg.org/calckey/calckey), #Forĝejo (https://Forgejo.org), and #NNCP (https://www.complete.org/nncp/) servers, and upscaling those virtual machines with additional HDD space or attaching attaching additional storage volumes from your provider is expensive, and geographically homed.

In that scenario, you could purchase a single storage VPS with a huge volume, deploy Garage with a few S3 buckets to accommodate your Fediverse, Forĝejo, and NNCP servers, and meet all of your storage needs very affordably.

If those VPSes are scattered about the globe in different data centers or providers, you can deploy additional Garage instances, distributing those existing S3 resources globally, and increasing availability while reducing latency.

I suppose you could also embrace #Cloudflare to address that last issue (I'm kidding, of course. Cloudflare == Satan).

I know you're primarily playing with a lot of #smallweb, #IoT, and Home #LAN based projects - a single Garage instance in the cloud could deliver your content at scale and speed.

#tallship

⛵

.

Cloudflare reports 2GB more usage than Mastodon... Has anybody built a tool to understand the discrepancy? would love to avoid writing it myself.

OTOH a lower hanging fruit is applying the patch to delete avatars and headers from the cache.

#mastoAdmin #s3 #r2

Hello #mastoadmin s,

Some of the remote images are marked as "not available" for me, but when I click it it's there in the origin. Do I have to increase the S3_OPEN_TIMEOUT and S3_READ_TIMEOUT values (the default is 5 secs) ?

I'm using #idrive as backend if that would matter.

#mastodon #s3

Anyone tried connecting fediverse projects to Storj?

Is it true? They claim: "Fast, secure cloud storage at a fraction of the cost."

#pixelfed #mastodon #misskey #pleroma #rebased #opensource #foss #fediverse #instance #microblog #activitypub #s3 #storage #storj #cloud

Oh yes mama!

#Publii fully supports #Cloudflare #r2 (with the custom #Amazon #s3 endpoint).

And when you attach your site to your Cloudflare domain your static site is now 100% served from Cloudflare with no origin server needed!

Hell yes!

• https://www.getpublii.com
• https://www.cloudflare.com/products/r2/

I think something has gone sideways with my Digital Ocean Space.

#DigitalOcean #S3 #Admin #MastoAdmin

Maintenant que #mastodon tourne ici, on va attendre un peu, voir ce que ça donne. Si c'est stable, et que mon infra n'est pas trop sous l'eau, on pourra envisager 2-3 #OrangePi 5, du NVMe, et monter un stockage #S3 via #minIO :). On verra en février, une fois les "bonnes nouvelles de fin d'année" passées....

So this happened. I managed to delete and/or lifecycle out some unwanted files on #Amazon #S3 so that in theory I am using less than 3 GB - ( although metrics show 685GB still) That is 682BG of hidden? files or something else that I don't understand.

Also this graph below shows number of objects has doubled.
I'm aware that lifecycle clearing out can take a few days but "didn't expect the spanish inquisition" to paraphrase an old joke. See graph below. I'm hoping to see a downwards trend :)

#mastoadmin sadly #backblaze b2 is banned in my country (most possibly for the stupidest reason the whole api domain got banned), so I'm looking for alternatives for #mastodon storage solution.

I've heard about #idriveE2 from #idrive . Has anyone used it, along with #cloudflare or similar CDN? How was your experience with #idrive #e2 ?

https://www.idrive.com/e2/

Or has anyone been using other storage solutions apart from #aws #s3 and #backblazeb2 ? E.g. #wasabi ?

https://micro.arda.pw/@arda/109463447161704765

Geek question here. I have 1.2GB of log files on Amazon S3 that I am trying to delete. I am being charged each month for these. I started using S3 in 2015 never imagined that logs would not rotate out.

I have tried deleting the logs and can seem to get about 400,000 of the suckers and then it stalls at about that level which is 400mb or more

Because of being billed in $US it all adds up. I'm trying to reduce my #Amazon bills #S3

Any tech especially in #NZ have any answers.

For my #mastodon server, I’m hosting all my images and static content on #aws #s3. It’s working great but growing in size pretty quickly. I know there are tools to prune stale content, but I’ve also added a S3 Lifecycle Policy to transition everything to Intelligent Tiering.

I’m migrating my #mastodon media storage from #AWS #S3 to #backblaze #B2 for cost reasons (if my maths is correct I should be able to cut my cost by about 82%).

But boy is it slow: it’s taken about 20 hours so far, transferring just 30 GB using rclone. 🙄

Thanks to to #nginx and #cloudflare there is at least no downtime though …

#mastoadmin #mastoadmins

@ericmann #aws RDS/Aurora, in our case. Storage and compute being separate makes it very easy to spin up replicas or failover. Prod clones for testing/dev are fast, but snapshot restores still take too long.

#mysql and #s3 are the only two parts of our stack that aren't ephemeral. Replicating those allows us to jump around regions.

A couple of days later and my 4GB #Linode is still handling #mastodon with ease. Looks like it might be overkill for a single-user instance, in fact.

The biggest concern seems to definitely be media storage — save yourself the headache and set up #S3 (or something compatible like Linode Object Storage, DigitalOcean Spaces, Wasabi, etc.) *before* your new server has the chance to federate with others! And don't forget a cronjob to purge media older than a week. 💸

#mastoadmin #mastoadmins