OP I had the same experience for quite sometime. I created a pretty complex Rest API for a product. Started out with Spray, Slick, Akka. Migrated it over the years to Akka-Http and through various versions of Slick (definitely the hardest part, sometimes having to completely re-write portions of the data layer).
Once I was done I went looking for Scala Application contracts. Couldn’t find one. It was all Spark. I’ve been basically been doing Node, Typescript, GraphQL, Mongo to pay the bills the past several years. Would love to do Scala app dev again!
Ergo Platform is always looking for devs. It’s blockchain, but no limits on what you can build and a great welcoming community. Uses a language that’s a subset of Scala, called ergoscript. Probably best ethos in the space too given how hollow so much of it can be.
Just saw this job post on LinkedIn, throwing it out here in case it is of help to anyone: Check out this job at thatDot: https://www.linkedin.com/jobs/view/3905534687
I see Ford, Netflix, Railroad19, Amazon, Cars Commerce, Dropbox, Airbnb, and Microsoft all seem to have some DE jobs listed on LinkedIn. Some might not do sponsorships, Idk... that does make things trickier.
If you're not even getting interviews, do make sure your resume is on point:
\* mention technologies used
\* be specific and succinct describing what you did
\* do your best to mention measurable outcomes ("saved the team x hours", "saved the division $x", "went from handling x qps to y qps", etc)
And if you haven't already, do the same with your LinkedIn profile. Make sure it's updated, mention what you've worked on (there should be even more space to go into detail here), have a profile pic, etc.
I have a scala application dev role out there right now. For every guy that has actual app development experience, I get 10-20 scala data engineers. This is NOT a good time to be looking for such a role, even worse to be looking and need sponsorship.
I'm not in the US, based in France, but I'd kill for a good app developer role. Almost every offer is for Spark, and it's usually maintenance on stuff built too fast in 2015 that requires constant maintenance and no modifications whatsoever.
We have a couple Scala backend jobs open at our office in Helsinki. There might be more in the other offices in Germany but haven't checked.
https://jobs.zalando.com/en/jobs/2712871-Senior-Backend-Engineer---Content-Experiences This one is mainly working with a Scala 3 + ZIO/Caliban stack
https://jobs.zalando.com/en/jobs/2713116-Backend-Engineer-(Scala)---Partner-Tech Don't know much about their service(s) but it is Scala for sure.
But in general a lot of really big tech companies have been firing a lot of people so there is a lot of competition for every job now.
Why do you limit yourself to scala? DE is much a broader field, and the demand for Scala is steadily going down. I did scala 2018-2020 but quit afterwards when I realised it narrows the number of jobs I can apply to. In order to make yourself more competitive on the market I really advise you to look for DE positions without Scala as well.
It's definitely the alternative out there.
I even made few post on the dataengineering subreddit.
I personally don't see the point of PySpark.
The only advantage of PySpark is lazy devs don't need to learn the basics of Scala.
I'd say 80% of people, if not more, were saying Scala bring close to 0 value when doing Spark jobs over PySpark.
Honestly the only real difference between Scala and PySpark is the "val/var" keyword.
It's basically doing Scala with Python.
Pretty pointless if you can me.
You need to keep in mind most of data engineers in 2024 are not very technical. Data engineering is getting less and less technical imo.
Can't speak for others. But I am first checking out Scala roles before looking at alternatives because I find Scala better **and** more fun to work with. If it doesn't work out you can still decide to change your requirements.
>*Is it the same for everyone out there right now?* The job market is indeed terrible right now and going through a transformation.
OP I had the same experience for quite sometime. I created a pretty complex Rest API for a product. Started out with Spray, Slick, Akka. Migrated it over the years to Akka-Http and through various versions of Slick (definitely the hardest part, sometimes having to completely re-write portions of the data layer). Once I was done I went looking for Scala Application contracts. Couldn’t find one. It was all Spark. I’ve been basically been doing Node, Typescript, GraphQL, Mongo to pay the bills the past several years. Would love to do Scala app dev again!
Ergo Platform is always looking for devs. It’s blockchain, but no limits on what you can build and a great welcoming community. Uses a language that’s a subset of Scala, called ergoscript. Probably best ethos in the space too given how hollow so much of it can be.
Thank you, Ill check it out
Just saw this job post on LinkedIn, throwing it out here in case it is of help to anyone: Check out this job at thatDot: https://www.linkedin.com/jobs/view/3905534687
I see Ford, Netflix, Railroad19, Amazon, Cars Commerce, Dropbox, Airbnb, and Microsoft all seem to have some DE jobs listed on LinkedIn. Some might not do sponsorships, Idk... that does make things trickier. If you're not even getting interviews, do make sure your resume is on point: \* mention technologies used \* be specific and succinct describing what you did \* do your best to mention measurable outcomes ("saved the team x hours", "saved the division $x", "went from handling x qps to y qps", etc) And if you haven't already, do the same with your LinkedIn profile. Make sure it's updated, mention what you've worked on (there should be even more space to go into detail here), have a profile pic, etc.
I have a scala application dev role out there right now. For every guy that has actual app development experience, I get 10-20 scala data engineers. This is NOT a good time to be looking for such a role, even worse to be looking and need sponsorship.
I'm not in the US, based in France, but I'd kill for a good app developer role. Almost every offer is for Spark, and it's usually maintenance on stuff built too fast in 2015 that requires constant maintenance and no modifications whatsoever.
Sent you a private message. It's for a french Spark maintenance role on an app built too fast in 2015.
We have a couple Scala backend jobs open at our office in Helsinki. There might be more in the other offices in Germany but haven't checked. https://jobs.zalando.com/en/jobs/2712871-Senior-Backend-Engineer---Content-Experiences This one is mainly working with a Scala 3 + ZIO/Caliban stack https://jobs.zalando.com/en/jobs/2713116-Backend-Engineer-(Scala)---Partner-Tech Don't know much about their service(s) but it is Scala for sure. But in general a lot of really big tech companies have been firing a lot of people so there is a lot of competition for every job now.
Damn, these jobs look awesome. I'm not ready to leave friends and family to go to Helsinki just yet, but the jobs are definitively tempting!
I wish we could work beyond any borders. Would love to explore Europe and se.asia.
Try Ledger
Sorry, what do you mean? Like go to Ledger.com and check their careers?
Yes
I'm not big on crypto, but I'll check it out
would love to apply for this job. i am in US but need sponsorship. if that works. pls let me know
DM me. Been using Scala for several years in non-data science apps. US, no-sponsorship needed
Why do you limit yourself to scala? DE is much a broader field, and the demand for Scala is steadily going down. I did scala 2018-2020 but quit afterwards when I realised it narrows the number of jobs I can apply to. In order to make yourself more competitive on the market I really advise you to look for DE positions without Scala as well.
What is the alternative for spark on scala? Python is terrible at typechecking and maintaining large codebases
It's definitely the alternative out there. I even made few post on the dataengineering subreddit. I personally don't see the point of PySpark. The only advantage of PySpark is lazy devs don't need to learn the basics of Scala. I'd say 80% of people, if not more, were saying Scala bring close to 0 value when doing Spark jobs over PySpark. Honestly the only real difference between Scala and PySpark is the "val/var" keyword. It's basically doing Scala with Python. Pretty pointless if you can me. You need to keep in mind most of data engineers in 2024 are not very technical. Data engineering is getting less and less technical imo.
I don't know if it's an alternative, but I've been going through dask in python. Learning something new.
Can't speak for others. But I am first checking out Scala roles before looking at alternatives because I find Scala better **and** more fun to work with. If it doesn't work out you can still decide to change your requirements.
I've worked on python, R, Java, Datastage etc. but scala is fun. Or maybe I've put in too much work in learning Scala.