The Scala Programming Language
Scala combines object-oriented and functional programming in one concise, high-level language. Scala's static types help avoid bugs in complex applications, and its JVM and JavaScript runtimes let you build high-performance systems with easy access to huge ecosystems of libraries.
- Seamless Java Interop: Scala runs on the JVM, so Java and Scala stacks can be freely mixed for totally seamless integration.
- Type Inference: So the type system doesn’t feel so static. Don’t work for the type system. Let the type system work for you!
- Concurrency & Distribution: Use data-parallel operations on collections, use actors for concurrency and distribution, or futures for asynchronous programming.
- Traits: Combine the flexibility of Java-style interfaces with the power of classes. Think principled multiple-inheritance.
- Pattern Matching: Think “switch” on steroids. Match against class hierarchies, sequences, and more.
- Higher-order functions: Functions are first-class objects. Compose them with guaranteed type safety. Use them anywhere, pass them to anything.
Shasta Tek in Scala
We have used Scala for the ETL process for one of our data-intensive reporting, involving complex queries. It provides a fast and highly reliable in-memory computation and efficient in interactive queries and iterative algorithm. We used SparkSQL for real-time analytics and Streaming data. We have used Scala o generate the graphical type of reports. We write a class program and execute the code that runs on JVM. it converts Scala code into Java byte code. Finally, we are creating the jar file to run the code into a Hadoop environment using Spark-Submit. Spark-submit shell script allows you to manage your Spark applications. while executing the spark-submit, the script first checks whether the SPARK_HOME environment variable is set and sets it to the directory that contains bin/spark-submit shell script if not then executes a spark-class shell script to run Spark Submit standalone application. Logging and Error handling will be separately stored as a file in the Hadoop system.