Wednesday, January 20, 2021

Book Notes: Extreme Ownership: How U.S. Navy SEALs Lead and Win

After reading rave reviews about the book Extreme Ownership: How U.S. Navy SEALs Lead and Win I decided to give it a go. 

This book deserves - no not deserves but - commands respect. It teaches various leadership principles and how to apply them in different situations. Authors Jocko Willink and Leif Babin - who were both U.S. Navy SEALs - teach us the leadership principles using real life examples from numerous combat operations executed in Iraq. 

This book is for everyone who is stepping into any sort of leadership position, in business or personal life or absolutely anything. 

Disclaimer: This post is by no means a summary of the book, I would encourage everyone to go ahead and grab the book and give it a go. I am sure you will learn a ton about leadership!

Book Notes

Extreme Ownership

When things go wrong - which they will - it is responsibility of the leader to take full and complete ownership. It's very easy to blame others for the failures, but it always results in suboptimal and dysfunctional teams. The buck stops at the leader, there is no one else to blame. 

This will ensure that the team is not busy playing "the blame game" and focuses their energy onto the things that matter and gets the job done. Once this principle gets embedded into your standard operating procedures, it becomes easier for the teams to talk more constructively about the failure, they will learn what did they do wrong and what should they do better in the future.

There are no bad teams, only bad leaders.

Leaders who want to achieve higher standards of performance for the team, must recognise - what really matters is "it's not what you preach, it's what you tolerate". 

If substandard performance is accepted and no on is held accountable - the poor performance becomes the new standard. Leaders must always keep learning and improving, they must also build this mind-set into the team.

Believe

In order for your team to accomplish a mission, the leader must truly believe it. If the leader doesn't believe in the mission, he or she will not be able to make the team believe in the mission. 

If the team doesn't believe in the mission then, the team is setup for failure. They will not take the risks required to overcome inevitable challenges necessary to execute the mission. 

Leave the Ego behind

Ego clouds and disrupts everything: the planning process, the ability to take good advice, and the ability to accept constructive criticism. The most difficult ego to deal with is your own! 

When personal agendas become more important than the team and the overarching missions success, performance suffers and failures is ensured. Admitting mistakes, taking ownership and developing a plan to overcome challenges are integral to any successful team and Ego is an enemy to all that.

Cover and Move: Teamwork

All elements within the greater team are crucial and must work together to accomplish the mission. Teams should mutually support one another for that singular purpose. Departments and groups within the team must break down silos. If the teams starts operating independently or work against each other, the results can be catastrophic.  

The leaders should continually keep perspective on strategic mission and remind the team that they are part of the greater team and the strategic missions is paramount. When different teams use the language "us versus them" it's a giant red flag. The leader must guide the team to overcome this mentality and work together, mutually supporting one another. 

Simple - KISS - Keep It Simple Stupid

The plan has to be simple. It should be simple enough so that each and every team member understands the plan and has complete clarity on it. This is essential because, when things go wrong and they always do go wrong, complexity compounds issues that can spiral out of control into a complete failure. 

It is critical that leaders enable the team members to ask questions about things that they do not understand about the plan or their responsibilities. Leaders must encourage this communication and take the time to explain, so that every team member fully and completely understands the plan. Without this failure is almost always guaranteed. 

Prioritise and Execute 

At times there are many important problems to tackle simultaneously. Even the most competent of leaders can be overwhelmed, if they try to tackle all of them simultaneously. Instead, leaders must determine the highest priority task and execute. When overwhelmed, fall back upon this principle: Prioritise and Execute. 

It is crucial for leaders to step back and maintain the strategic picture. This is essential to help them correctly prioritise. These are the steps to do it effectively

  • Evaluate the highest priority problem
  • Clearly state the highest priority effort for the team.
  • Seek inputs from key leaders and build a solution 
  • Execute the solution, focus all resources towards this priority task.
  • Move to the next problem. 
  • Repeat.
  • When priorities change, make sure to pass the information up and down the chain.
  • Keep an eye on other problems that are building up.
Decentralised Command

Humans are generally not capable of managing more than six to ten people, particularly when things go sideways and inevitable contingencies arise. Teams must be broken down into manageable smaller groups with a clearly designated leader. Subordinate leaders must be empowered to make the decision on key tasks necessary to accomplish that mission in most effective and efficient manner possible. 

This doesn't mean that the subordinate leaders or the team members operate on their own, this will result in chaos. Instead, they must fully understand what is within their decision-making authority i.e. the left and right limits of their responsibility. They must also communicate with senior leaders to recommend decisions outside their authority and pass critical information up the chain so the senior leadership can make informed strategic decisions. Subordinate leaders are expected to plan and execute rather than ask "What do you want me to do?" To get to that stage, each team member must clearly understand the strategic mission. 

Plan

A broad and ambiguous mission results in lack of focus, ineffective execution and mission creep. To prevent this the mission must be carefully refined and simplified so that it is explicitly clear. Mission must explain the overall purpose and desired "end state".

Leaders must delegate the planning process down the chain as much as possible to key subordinate leaders. Team participation even from most junior personnel is critical in developing bold, innovative solutions to problem sets. Giving the frontline teams ownership of even small piece of the plan gives them buy-in, helps them understand the reasons behind the plan and creates the belief in the mission. This translates into far more effective implementation and execution. 

Leading Up and Down the Chain of Command

A good leader is immersed in the planning and execution of tasks, projects and operations to move the team towards a strategic goal. Such leaders possess insight into the bigger picture and why some tasks need to be done. This information does not automatically translate to subordinates. It is paramount that senior leaders explain to the subordinate leaders and the team how their role contributes to the big picture success. This understanding helps the team members prioritise their efforts in rapidly changing, dynamic environment. This is called leading down the chain of command. 

If your boss ins't making a decision in a timely manner, don't blame the boss. First, take extreme ownership and introspect what you can do to better convey the critical information for decision to be made? Leading up the chain of command requires tactful engagement with immediate boss to obtain the decisions and support necessary to enable your team to accomplish the goal. To do this, leader must push situation awareness up the chain of command.

Decisiveness amid Uncertainty

The leaders cannot be paralysed by fear. That results in inaction. It is critical for a leader to act decisively amid uncertainty. To make the best decision they can based on only the immediate information available. Leaders must be comfortable in chaos and act decisively. Waiting for 100% information before making a decision is catastrophic. 

Discipline Equals Freedom - The Dichotomy of Leadership

Leadership requires finding the equilibrium in the dichotomy of many seemingly contradictory qualities. 
  • A leader must lead but also be ready to follow.
  • A leader must be aggressive but not overbearing. 
  • A leader must be brave but not foolhardy
  • A leader must have a competitive spirit but also be gracious loser
  • A leader must be attentive to details, but not obsessed by them
  • A leader must be strong but likewise have endurance, not only physically but mentally.
  • A leader must be humble but not passive; quiet but not silent. 
  • A leader must be close to subordinates but not too close. Leaders must never get so close that the team forgets who is in charge.
  • A leader must exercise Extreme Ownership and decentralised chain of command by giving control to subordinate leaders.
  • A leader has nothing to prove but everything to prove!
Conclusion 

I learned a great deal from this book, it has a world of experience to offer. I hope, you are inspired enough to pick this book up and give it a go!

Sunday, November 29, 2020

Book Overview: High Output Management

I finally got my act together and read the book High Output Management by Andrew Grove - The legendary ex-chairman and CEO of Intel. 

This is by far the most important book to read for improving the output of your company. The book applies production principles to management. It also provides comprehensive overview of a managers role and how they can improve the output of their own organisation. The main thing I learned from the book was:

Managers Output

A manager’s output = the output of his organisation + the output of the neighbouring organisations under his influence.

A manager's objective is to increase the output of those below and around him. A manager should therefore focus on high-leverage activities that have a multiplicative impact on the overall output of his subordinates and peers.

The book states that, “When a person is not doing his job, there can only be two reasons for it. The person either can’t do it or won’t do it; he is either not capable or not motivated.” Hence, to increase the performance of an individual, manager has only two options. 

  • Training 
  • Motivation
To increase motivation, manager needs to understand an individual’s highest level needs. The needs can be mapped to the Maslow's Hierarchy of Needs

Training is the highest leverage activity a manager can do to increase the output of an organisation. 

If a manager spends 12 hours preparing training for 10 team members that increases their output by 1% on average, the result is 200 hours of increased output from the 10 employees (each works about 2000 hours a year). Don’t leave training to outsiders, do it yourself.


Meetings

A lot of time I have seen people end up with endless meetings with participants having no say in whats going on. This is very wasteful for both employee as well as the company. People's time should be treated as one of the most important asset in any company. Meetings should be purposeful and well-executed. The book categorises the meetings into two main categories. 

  • Process-oriented meetings
  • Mission oriented meetings.
The process oriented meetings include the one-to-one meetings, staff meetings and performance review meetings. 

The book goes to great length to explain the importance of one-to-one meetings. They are not only a fundamental element in the manager/employee relationship, but also act as the best source for organisational knowledge that a manager can get. 

Performance reviews are easily mistaken as simply a way to assess performance and evaluate compensation. The fundamental goal of a performance review is to improve the subordinates performance. The performance review is intended to influence a subordinate's performance, this makes this activity one of the manager's highest-leverage activities. Great care needs to be taken in the preparation and delivery of a performance review.

The mission oriented meetings are created on an ad-hoc basis to take a decision. These type of meetings should be lesser than 25% of the total meetings. The meeting attendees should be well prepared and all people vital to taking the decision should be part of the meeting.


Decisions

Last point I want to mention from the book is about making a decision.

Managers technical knowledge will become dated over time, so decision-making in a technical, information-driven environment needs to be a process that takes into the account both people with knowledge-power and people with position-power. 

People with knowledge-power have the knowhow of how the technology works, they understand the technology better. People with position-power are the ones who are responsible for actually taking the decision.

To make any decision all of the below mentioned questions should be answered in advanced:

What decision needs to be made?
When does it have to be made?
Who will decide?
Who will need to be consulted prior to making the decision?
Who will ratify or veto the decision?
Who will need to be informed of the decision?

The book offers so much guidance, I recommend reading the book multiple times to actually digest and implement the advices fully.

Saturday, October 31, 2020

Book Overview: The Art Of Game Design

Well, where do I begin. The Art Of Game Design by Jesse Schell is the holy grail for game designers. It is simply a must read for anyone who is serious about build a game.

This book is extremely detailed and it will be foolish of me to summarise this book here. I will instead provide an overview about the book and encourage you to read this awesome book. The kindle version of the book can be bought from Amazon

Book Overview

The book starts by listing down the skills that a game designer needs to learn in order to design an awesome game. It lists around 20 essential skills, ranging from Anthropology, Architecture, Engineering, Economics, Mathematics, Sound and Music, Psychology to Visual Arts.

After reading this list for the first time, my respect for the game designers sky rocketed. I was wondering, can any single person have all these skills? 

As with everything else the most important skill for a game designers is "Listening"

Game designers must listen to many things. These can be grouped into five major categories: team, audience, game, client and self!

The book then starts dissecting the process of creating a good game. It's like peeling an onion, layer after layer the book talks about various important aspects to consider while designing a game.  Here's a comprehensive but elegant map of things involved in creating a good game design 


The book gives out a set of questions that must be asked by the game designer to iterate on the current game design to make it better. These questions are called lenses. 

There are about 100 lenses described in the book. It also gives a set of most important question that the game designer must ask to look at the current game design from a given lens. For e.g. "The Lens of Fun" the questions for this lens are 

  • What parts of my game are fun? Why?
  • What parts need to be more fun?

Using the knowledge from the book we improved many things in our game design process. I would like to mention one very simple change we did which resulted in giving us great results. 

The book talks about "Finding a Brainstorming Partner". 

Finding the right brainstorming partner can make a world of difference-sometimes the two of you can get to great solutions many times faster than either of you could alone, as you bounce ideas back and forth and complete one another's sentences.

 We implemented this simple change in our process and we saw immediate improvements in our progress. The book is filled with tips like these. These tips can have material impact on the final game.

The book talks about 100 different lenses, obviously it's difficult to remember all of them. To make the lenses more handy and easy to use, author has an iOS and Android app called The Art of Game Design: a Deck of Lenses

Overall, I feel it was a great read and I totally recommend reading this book. If you have built a game in the past or are considering building a game just pick up a copy today.

Wednesday, September 30, 2020

How To Invoke Swift code from Objective C code

I wasted a couple of hours trying to do exactly this. The lack of clear documentation didn't help either. I decided to write small post about this so that others do not have to waste their time digging through the documentation.

How do you invoke Swift code from Objective C code?


Swift class either need to be derived from NSObject or they need to be @objc attribute. Let's look at a simple swift class that we want to invoke from Objective C


When you add the Swift class to the Objective C project, XCode will ask whether you want to generate the "Bridging Header", just say yes. This header is useful when you want to invoke Objective C code from Swift code. But we are interested in doing the opposite, i.e. invoke Swift code from Objective C code.

Next we need to head to XCode -> Project Properties -> Target -> Build Settings, search for "Generated Interface". You should see a single entry called "Objective-C Generated Interface Header Name". You need to make a note of the generated header file name, it follows the format "<Product Module Name>-Swift.h"




This is the name of the header file that we need to include in our Objective C code to invoke the Swift code. This header file is auto-generated by XCode.

In the Build Settings search for "Bridging Header". You should see an entry called "Objective-C Bridging Header". This setting points to the path of the Bridging Header generated by XCode earlier. The value here should have been populated by XCode automatically. 


We also need to set the value of "Swift Language Version" Build Setting. Search for "Language Version" and set the value.



Now that the Build Settings are out of the way, here is the sample Objective C code which invokes the Swift class we defined above.


Please note that you can only include the "<Product Module Name>-Swift.h" in the .m or .mm files. To use the Swift class in .h file, just forward declare it using @MySwiftClass.

Thats about all there is to invoke Swift code from Objective C, we do not need to change any other Build Settings. If the documentation was clear enough, I would not have to write this post!

Sunday, August 30, 2020

Invoking AppStore Connect API via Command Line

Sometime back Apple had opened up the AppStore Connect API for its developers. The API is a bunch of RESTful service which can be used to customise and automate our workflows. It helps us automate tasks across developer tools, such as App Store Connect, Xcode, and Certificates, Identifiers & Profiles, to give us greater flexibility and increased efficiency in our workflows. We could use it for development, beta testing, managing app metadata, reporting etc.

It can be very helpful to invoke the AppStore Connect API via command line. I use it primarily to check the structure of the response object. Although the documentation of the AppStore Connect API is good, but there is nothing as good as seeing the response object with real values in it.

In this post we will see how can we invoke the AppStore Connect API from the command line.

How To Do It? 

To invoke the AppStore Connect API via command line we will follow these steps.

  • Create API Key from the App Store Connect Web Portal
  • Create JWT JSON Token for Accessing API using XCToken
  • Invoking the API from command line using curl.

Create API Key 

  • To generate an API Key you will need to login to iTunes Connect portal. 
  • Navigate to the "Users and Access" section


  • In there, navigate to the "Keys" tab
  • Choose the "Key Type" as "AppStore Connect API"
  • Make a note of the "Issuer Id" we will be using this later on to generate the JWT JSON Token.


  • Generate a New API Key, by clicking the "+" icon. This will open up a dialog which ask you to name the key and select the Access type. I am going to choose "Developer" for this post.


  • This will generate the API key, make a note of the generated Key Id we will be using it later to generated the JWT JSON Token.
  • iTunesConnect will give you an opportunity to download the generated key only once. So please make sure you download it and keep it safe.


  • This completes the steps necessary to generate an API Key.

Create JWT JSON Token

To Generate the JWT JSON Token we are going to use a command line utility called XCToken. It can generate on-demand JWT tokens forAppStore Connect API. To install the utility just run the following command.

gem install xctoken
To generate the JWT JSON Token this utility expects three environment variables.
  • ISSUER_ID = This is the issuer id which we have noted in the earlier step from the iTunes Connect page.
  • KEY_DIR = The full directory path where the API key was downloaded from iTunes Connect
  • KEY_ID = The key id of the newly generated API key.
Once these environment variables are set up the XCToken is ready to generate the JWT JSON Token. Here is the sample script that will generate the token.

export ISSUER_ID=ABCD
export KEY_DIR=~/Downloads/
export KEY_ID=ABCD1234
xctoken  generate
This will spit out the token on the console, make a note of this token, we will be using it to invoke the AppStore Connect API.

Invoking the API

Now that we have generated a new JWT JSON Token, we are ready to invoke the AppStore Connect API. Here you can find various endpoints and their documentation. For e.g. to get a list of all your apps, use the following command.

curl  https://api.appstoreconnect.apple.com/v1/apps --Header "Authorization: Bearer <GENERATED TOKEN>"
Thats about all we need to do to invoke the AppStore Connect API via command line!

Friday, July 31, 2020

How To Reduce the Disk Space Need for Amazon Redshift - Part 2

This post will conclude the, process of reducing the disk space need for Amazon Redshift. If you haven't already read Part 1 in this 2 part series, I strongly recommend that you go read it now, I will wait!

Right, so now that we know why we are doing what we are doing, let's get straight to the point. How to actually do it?

How To?


We will use an open source library called Spectrify. It basically helps us with the following
  • Export a Redshift table to S3 (CSV)
  • Convert exported CSVs to Parquet files in parallel
  • Create the Spectrum table on your Redshift cluster
It basically performs all the steps needed to get our table offloaded from Redshift and then setup as a spectrum table on the redshift cluster. 

Since the entire process is network and CPU intensive, its advisable that we do it from an Amazon EC2 instance - t2.xlarge is highly recommended. Assuming that you have an Amazon EC2 instance, heres the script gets the job done.


The script is designed in such a way that it installs all the necessary dependencies required for its execution.

It basically performs the following steps:
  • Ensure that we have psql installed. Thats needed so that we can execute the commands on Redshift database. Install it if required.
  • Create the schema if required. It needs to be done only once. We will host our spectrum tables in spectrum schema.
  • Check if spectrify is installed. Install it with all the necessary dependencies, if required.
  • Export the data in the Redshift tables to CSV files on S3.
  • Convert the CSV files to Parquet format.
  • Moving the files to appropriate move path, so that we can support incremental exports.
  • Create the spectrum table in Redshift. Don't need to create the table again and again. Only do it once.
  • Truncate the Redshift table if required.

Once the script finishes running, you should have your spectrum table ready to be queried and used like any other Redshift table. It will have the same structure and data as the original Redshift table!

What about Incremental Offload?


Totally possible! We can run this script again and again on any table in Redshift. It will keep appending data to the spectrum table without overwriting the earlier data. If a table was offloaded earlier, running it again on the same table, will offload only new rows that got added after the last run.

I hope more and more people are able to use of this awesome feature and reduces the disk space need for Amazon Redshift cluster!

Tuesday, June 30, 2020

How To Reduce the Disk Space Need for Amazon Redshift - Part 1

At Makkajai, we use Amazon Redshift as our data warehouse. Redshift has got good features, but when it comes to providing disk space its a bit expensive. Especially, if you choose the DC kind of a cluster. For e.g. the dc2.large comes with only 160 GB storage. 

With any data warehouse, the data constantly keeps increasing and we always run into disk space related issues.

One of our cluster recently ran out of disk space. I started looking around for possible solutions to get around this problem.

Spend More Money

One very obvious alternative was to Scale the Amazon Redshift cluster horizontally.

This means adding a new Amazon Redshift nodes. While this solution is pretty easy, it comes with cost. An additional $140-$190 (approximately) per month + taxes. 

Also the data will keep piling up and we will have to keep adding new nodes to increase the cluster capacity. Hence, this approach becomes more and more expensive over time. 

Offload the data somewhere?

Theoretically, we could offload a lot of data to a cheaper storage (for example Amazon S3) and use that for querying data from Redshift. But we need to make sure that the data is still queryable from Amazon Redshift, just like we query any other table - that would be perfect. 

I looked around to see, if there is was a way to get this done with least amount of pain. Thankfully, we found Amazon Redshift Spectrum

It is build for specifically this use-case. 

... Being able to query data stored in S3 means that you can scale your compute and your storage independently, with the full power of the Redshift query model and all of the reporting and business intelligence tools at your disposal. Your queries can reference any combination of data stored in Redshift tables and in S3.

When you issue a query, Redshift rips it apart and generates a query plan that minimises the amount of S3 data that will be read, taking advantage of both column-oriented formats and data that is partitioned by date or another key.

But how much do we pay extra?

Spectrum pricing is based on the amount of data pulled from S3 during query processing and is charged at the rate of $5 per terabyte (you can save money by compressing your data and/or storing it in column-oriented form). You pay the usual charges to run your Redshift cluster and to store your data in S3, but there are no Spectrum charges when you are not running queries.

I think this paragraph summarises it pretty well.

What do we eventually get?

Using this approach we were able reduce the Disk Space needed for Amazon Redshift significantly. We were nearing 90-100% earlier, after offloading few bigger tables we are at 65-67% disk space usage.

In the next post, I will document the exact steps needed to get this setup in place. 
Have some Fun!