there is a reason why people develop for S3: a lot of enterprise data is there. people ingest there from various sources. and it's not just parquet usually, it's multivendor sources writing to an iceberg catalog.
nobody will run minio on AWS other than hobby projects and small demos.
I regularly work with iceberg datasets in the double digit TB range per dataset. keep that in mind when you think about sizes. databricks, snowflake, large enterprise vendors: they are targeting these sizes.
they exist, sure. And I'm sure it can handle PB+. on prem is an existing market, however, if you reread my comment I talked about running minio on AWS because S3 is too expensive - just doesn't make sense to do.
I've yet to met a Fortune 100 who isn't mostly using either on prem or a large hyperscaler (S3/Azure/GCS).
The large enterprise vendors are not prise-sensitive. They're on AWS because you never get fired for picking AWS, and there isn't really any other choice for these vendors regardless of AWS ripping you off.
At this point S3 is a standard interface. All sorts of cloud providers and open-source projects provide S3. If you're on AWS, price isn't the reason. You pick AWS because you don't see your company taking a risk with anything else.
S3 doesn't mean expensive. AWS does. But AWS users are fully locked-in, they'll pay whatever the price is.
There is a scale between prise-sensitivity and risk-averseness, from my point of reference large companies are much more risk-averse than they are price sensitive. Of course this will vary, CTOs exist in all sort of different environments.
Price is not the reason people chose AWS. Some companies use Azure. The current startup at $WORK uses yet another smaller Cloud. And yet AWS sill has the clear lead in market share. That's because price is far from the only factor, and not even the main factor.
> There is a scale between prise-sensitivity and risk-averseness, from my point of reference large companies are much more risk-averse than they are price sensitive.
That's not true. It's just the way things work "saving money" isn't part of the KPI. Enterprise teams get a budget. If you "saved" you don't get it back. So unless there's a legit need it's ALWAYS easier increasing than cutting it.
It's not about risk. It's about power. They are price sensitive but in a way that doesn't matter to the bottom line i.e. if I can cut my AWS storage bill by 10% and then spend it on random tokens I'd do it.
They definitely havent. Tech side of companies is a Cost Center. And the main question the CEO/CFO makes to the CTO every week is "how can we reduce our AWS bill?" , even before the how was your weekend ? One.
there is a reason why people develop for S3: a lot of enterprise data is there. people ingest there from various sources. and it's not just parquet usually, it's multivendor sources writing to an iceberg catalog.
nobody will run minio on AWS other than hobby projects and small demos.
I regularly work with iceberg datasets in the double digit TB range per dataset. keep that in mind when you think about sizes. databricks, snowflake, large enterprise vendors: they are targeting these sizes.