I'm an intermediate Scala user. Scala compile times hurt my productivity significantly. Our best Scala developer says his approach is "I know the language so well, I usually write the whole feature in one go before compiling. That way I avoid the slow compile times."
Which feels to me like you will miss one of the advantages of having a compiler in the first place - that it helps you write correct code.
To be fair, in an IDE you do get some of the compiler feedback in real time. But not for the whole code base, if you're working on things that have high coupling with other modules then it can be really painful. (I'm looking at you, Spray serialization!! Ugh)
Depends on the size of the code base, how good your incremental compilation is, and what percentage of the code changes frequently. At Twitter the values are: large, great, and relatively high.
I've seen a lot of complaints about Scala's compilation speed (I believe it's either the 1st or 2nd most common complaint), but I've not really had any issues myself.
Could people who have had issues with compilation speed explain their workflow, I'd like to better understand the problem.
For example, here's my general workflow:
1. code in IntelliJ until the feature/fix is complete, and there aren't any red-wavy lines in my source tree
2. run `compile` in an already running sbt session.
As a result, I only find myself "waiting" for a compile a few times a day. And even then, because I'm using the incremental compiler it's usually only a few seconds.
The only reason I run the sbt compile at all is because the IntelliJ compiler is known to be a bit buggy, especially around things like implicit resolution and some of the more advanced features used for abstract programming.
I have seen CI builds take some time, but even then, in every project I've worked on, the complete compile-time is dwarfed by the time taken to run tests.
I'm not trying to dismiss problems that others have; I would love to learn more about them!
Here is one that I run into frequently. You're building a CRUD app, but want some semblance of sane FP interaction with your database. So you use Doobie [1] because free monads and all.
So you type your query using a Doobie SQL string interpolator...
sql"SELECT ItemId from dbo.Item WHERE IsOnSale = TRUE"
...but then you forget all of the ".query[Int].vector.transact(tx).unsafePerformSync" crap that you are supposed to tack onto the end of the string to actually run your query. What are your options?
1. Type a '.' character in your IDE and watch it crash while the presentation compiler thrashes around, sifting through a combinatorial explosion of implicit values (or whatever it is that makes it so slow). You quickly learn to stop using autocomplete. In a sane world, this should take milliseconds and you should quickly get a list of methods to guide you down the right path.
2. Type a '.' and try to recompile. Wait 30 seconds. While you're waiting, what work can you possibly do??? None.
3. Stop what you're doing, open up The Book Of Doobie [2], try to remember which page has the syntax that you need for doing this one simple thing, go to that page, navigate to the right part of the webpage, read the crap, think a bit, copy/paste it into your code. This FEELS more productive than (2), but is it really?
This is just one example, maybe not terribly compelling, but it happens. People (well at least me) have a limited amount of memory for administrivial crap to hold in our heads while we try to get our work done. Tooling is supposed to help with that, but with Scala it really starts to get in the way. Flow state just isn't possible.
I have heard anecdotes of developers running two or more instances of scalac so that they can work on more than one feature at a time in order to not completely waste away their time. Imagine the context switching there, and having to pay attention to the little oven timer and switching context when one compiler is finally done recompiling the same code it already compiled hundreds of times.
Thanks! I haven't used doobie yet (though I intend to), I had no idea it's API caused IntelliJ such an auto-complete headache.
I'm curious how a faster scalac would help in this case though. Perhaps I'm missing something, but scalac's errors don't really facilitate this kind of API exploration.
I completely agree that exploring an API (because who wants to memorise the standard library and API of all your dependencies) in IntelliJ is often quite cumbersome. One trick I've come to lean on a lot is to type '.ensur' to determine the type of the current term; this causes IntelliJ to present the method signature for 'ensuring', which is available (implicitly) on everything (except Nothing), showing the current type in its return type.
FWIW, the author of Doobie doesn't even use an IDE, which is par for the course I believe for users of Scalaz, Cats, Shapeless, etc. inference heavy libraries.
Abstraction isn't free in Scala, you pay for the features used; tooling suffers as a result, thus projects like Twitter's RSC come into being.
You may want to checkout Quill, or Perhaps Slick (though the latter I suspect is similarly IDE challenged). Barring that, give your IDE loads of RAM.
They're not a complete non-issue. They're just much less of an issue than not using Scala.
(And bear in mind that Twitter has literally the largest Scala codebase in the world. Almost all scala programmers work on something an order of magnitude smaller, with correspondingly smaller compile times)
One of the huge benefits of strongly typed language is the ease it introduces to managing large code bases. Now imagine google size monorepo and scala compilation speed
It depends on project size, twitter likely have some really huge scala project, but many organization are now going for microservices, so compile time is less of an issue
