Nyano @greenNayan

I love doing codewars problems. They really help solidify concepts and help me learn the language. But I can't count out SQL problems in HackerRank. Those are gems too. SELECT problems FROM sql ORDER BY rank DESC; : ) #programming #coding #SQL

Currently solving a coding problem. Got to convert a sequence to a formula. This is to make sure the algorithm is fast and also to make sure that the runner doesn't time out and be sent to oblivion or the shadow realm. : ') #programming #codewars #coding

Did a coding challenge where I used dictionary/hash-maps instead of regular count. Dictionary/hash-maps ran like a cheetah, whereas the count method ran like a rabbit. #programming #codewars #coding

When a new timeline or a chapter begins, I seem to always make git related jokes. Such as: git commit "New timeline initiated" git commit "Initiate new chapter" git push I really did love both choices. But a choice had to be made. #coding #programming #git

Using Jupyter Notebook for quick data inspection and analysis makes the process smoother and much more convenient. #coding #data

Raw data? Got to check for duplicates, if there are any null values, column values that do not make sense, formatting issues and many more. Got to make sure the data quality is good before injecting it into a database. #coding #dataengineering

The code you write in development isn't going to work perfectly when you deploy it in production. Got to tweak parts to make it work. In the progress of trying to make it work, "This worked perfectly in development,so why isn't it working?" might creep in : ) #coding #programming

Stitch, Fivetran and Airbyte are tools that you can use to easily injest your data from different sources.

Playing around with pyspark for the first time. Took me a while to realize you can add in multiple spark.jars packages by separating them with commas. Brain fart inc. Got to make sure you have the right package version to get your database connectors set up. #coding #apachespark

Currently learning data modeling in MongoDB. Embedded model: Denormalized data that capture relationships between data using a single document structure. Reference model: Normalized data. Using references such as id which link one document to the next. #coding #MongoDB

Setting up a development environment to test out your code and database saves so many headaches later on in the process. #programming #coding

A decoupled architecture allows debugging and adding new features to be simpler. Especially decoupling the ETL process, so one would know for sure where the error is occurring. #coding #dataengineering

Docker allows you to build a containerized application. This build can be shared with others and allows them to quickly build the specific version in their machine even if they use a different operating system. Helping to quickly get started and start building. #coding #docker

Big data is about find meaning from the data which is coming in at a rapid pace, from different sources and in wide range of formats. #coding #data #dataengineering

Terraform and serverless framwork allows you to set up infrastructure as a code. You will need to write some code in ymal file and it will set up your desired architecture. #coding #cloud

Data Normalisation means to efficiently organise data in a database. Having the data normalised is making sure you don't have redundant data, this reduces the storage that is being consumed. #programming #database

ETL vs ELT ETL is good for planning and structuring before loading the data into a data warehouse. ELT is good if you want to jam every data in a data lake and then structure it afterwards. #DataEngineering #coding #programming

Horizontal scaling: More machines Vertical scaling: More powerful system resources (CPU, RAM) #coding #programming #cloud

Good documentation can be a holy grail when learning something new. Cobined with personal research from multiple articles and resources is a perfect two punch knockout combo. #coding #programming

Using regular for loops seems cleaner, easier to break down problems and simpler to explain than using a one-liner list comprehension. But sometimes having a one-liner just looks super cool. #coding #programming