Tuesday, May 29, 2018

How To Print 1x1 Shipping label on a 2x2 A4 Generic Sticker Paper

When you are running a startup, you will face numerous business problems on a daily basis. Some problems are within your core competency and some fall outside your comfort zone. Being a startup founder, you really cant afford to not solve the problem because it falls outside your comfort zone. Also at times, you only need to be street smart to solve the problem and move on :)

I recently faced one such, not so interesting problem, but it was essential for me to solve it.

Background

We ship books to our customers in India and we recently moved to Delhivery as our delivery partner. Have you noticed the stickers on the packaging, when you receive deliveries from Amazon/Flipkart/Delhivery etc? Those stickers are called Shipping Label. It has bunch of information like:

  • Who is the package for.
  • Where is it coming from
  • Contact details of the client 
  • Contents of the package and its approximate value
  • And many other things.
This is how a Shipping Label looks:

When we create the shipment in Delhivery portal, they generate the shipping label for us. Shippers are supposed to print it and affix it on the shipment. So far it feels normal and business as usual, what's the problem is not clear?

The Problem

We usually we ship in bulk, hence we ship to hundreds of our customers in one batch. Delhivery generates a PDF with 1 shipping label on every page, hence if we are shipping 100 shipments, Delhivery will generate the PDF with 100 pages i.e. 1 label on each page in a 1x1 format.

If we had access to a specialised printer which could print these stickers on a sticker roll, that we would be sorted. But unfortunately we didn't have that printer.

There are generic sticker papers available in the market to print shipping labels. However each A4 size generic sticker paper would cost around Rs. 5. Its not optimal from cost as well as resources perspective to print just one sticker on the entire A4 page.

Considering the size of the Shipping Label, we could easily print 4 shipping labels on one A4 size paper. If we were able to do it, the cost of printing one label will drop 4 folds. Something like this


Initially you might think, why is saving few bucks so important. Thats because of a simple concept called unit economics. If you ever want to get your startup in the successful zone, you need to get the unit economics right :D!

The Solution

Now that we know whats the problem and why we need to solve it, lets focus on how did I get it done.

Speed of execution is everything in startup world. I had to solve this in way that its easily doable by any non-tech operations guy, at the same time I didn't have the luxury to build a sophisticated custom solution. 

So, what did I do? I tried to breakdown the problem into smaller steps and try to solve each of those smaller steps.

  • First thing I observed in the Shipping Label's PDF was that, There was some extra information around the shipping labels like the footer of the page and some more unimportant stuff. 
  • In order to arrange it in 2x2 format I need to trim the unimportant stuff. For this I looked for a site that could help me trim all the PDF pages in one go. Sejda was perfect for this. The free plan has some restrictions but we could live with those.
  • You could upload the PDF and you could crop all pages with a mask in one go. What I got after that was a PDF with all pages having only the important stuff.
  • Next, I need to export each page as a separate image either in PNG or JPEG format. This was necessary so that I could use mail merge to actually arrange the shipping labels in 2x2 format.
  • I exported the PDF pages to png using pdf2png site. The result was a zip file with all the PDF pages exported as PNG files.
  • Final step was to use "Microsoft Word Mail Merge" to arrange these images in 2x2 grid. I followed the this nice article, to get that done.
  • Once I followed the steps, I finally got a 2x2 grid of all my shipping labels. These could be printed on a generic A4 paper with a 2x2 sticker grid.
  • In the end, I had a very low tech solution (which could be easily followed by any Operations guy) to a business problem.
The entire solution is in line with the theory suggested by Mark Watney, from The Martian :)
You solve one problem... and you solve the next one... and then the next. And If you solve enough problems, you get to come home!


VoilĂ , my job here was done!

Friday, April 27, 2018

How to get Snowplow-Mini running on AWS

While looking at various Analytics engines we came across Snowplow Analytics. We wanted to give it a shot and experience it first hand. Luckily, they have something called as Snowplow-Mini. Its an easily deployable, single instance version of Snowplow. It essentially gives us, a taste of what Snowplow can do for us, as far as data collection, processing and analytics is concerned!

We started with the quick start guide and usage guide, performed all the steps mentioned there to get the Snowplow-Mini instance working. However, we did faced two annoying issues, investigating and fixing them, wasted a few hrs. This post is about those two issues, so that my fellow developers do not have to waste any time on investigating and fixing them.

Unable to: Generate a pair of read/write API keys for the local Iglu schema registry

We followed all steps mentioned in the usage guide but we were unable to generate the keys.
  • Navigate to http://<public dns>/iglu-server
  • Input the super API key set up in previous step, in the input box in the top right corner
  • Expanded the keygen section
  • Expanded the POST /api/auth/keygen operation
  • Input the appropriate vendor_prefix for this API key
  • Click Try it out!
At this, it should have generated the read and write keys for us. But all it did instead was, showed a progress bar and runs forever without return.

Investigating it in Chrome Developer Console revealed that the calls were failing with 401 UnAuthorized. After googling for this error a bit, I found that someone else was also facing a similar problem. Their solution was to do HTTP POST via CURL and that seemed to work. However it didn't work for us either.

I looked around for ways to debug the problem.
  • I connected to the Showplow-Mini instance via SSH (refer to AWS documentation on how to do this)
  • Checked the config under "snowplow" directory on the instance. Could not spot anything unusual there -- not that I knew much about it anyways :D
  • Checked the logs under "/var/logs" directory. Found a few things but could not really solve the problem.
  • Connected to PostgreSQL DB on the instance using the following command
    • psql --host=localhost --port=5432 --username=snowplow --dbname=iglu
      # Password is "snowplow"
  • Ran the query to check the API key
    • select * from apikeys;
  • What I saw next, made my jaw drop, in disbelief!
  • They API key is case-sensitive and the key Snowplow-Mini had saved was all in lowercase, even though when I had given it the key, I had given it in all caps.
  • Passing the key in small case and making the following call did result in generating the read/write API keys for local iglu schema registry
    • curl http://<IP address of your server>/api/auth/keygen -X POST -H "apikey: <your case sensitive API key>" -d "vendor_prefix=com.makkajai"
  • Duh! Yea I know.

  • How to connect to PostgreSQL Snowplow-Mini DB, I got to know that from here 
I must have easily wasted an hour trying to fix this problem. I hope others can save that time!

Unable to: See events in Kibana Dashboard

This was a tricky one. After raising sample events, I was unable to see them in Kibana Dashboard. This happens mainly because the "snowplow_stream_enrich" is not able to connect to the "elastic search service".

How Did I figure it out?
  • ssh into the Snowplow-Mini instance
  • I checked the logs under "/var/logs" directory. 
  • The logs seemed to be filled with exceptions like
    • Exception in thread “main” java.net.UnknownHostException: ip-xx-xx-xx-xx: ip-xx-xx-xx-xx: unknown
  • Googled it a bit, found the solution here 
  • Edit the file "/etc/hosts" and add the IP address information in that file as follows.
    • sudo vim /etc/hosts 
    • xx.xx.xx.xx ip-xx-xx-xx-xx localhost
  • xx.xx.xx.xx being the AWS local IP address.
  • Save and exit and re-start all services from the Snowplow-Mini console.
  • Generate a few events and open Kibana dashboard, and it worked this time!
After these two problems were out of the way, my Snowplow-Mini instance was fully up and running on AWS!

Saturday, March 31, 2018

How to Debug A Pre-built APK

Android Studio 3.0 added a nifty little feature -- The ability to debug and profile pre-built APK's. For developers working with a mix of Native (C/C++) and Java code for their applications, this is an extremely valuable feature.

I stumbled upon it, while I was looking for something else. Wrote this post with the intent that more people will be able to find this feature and make good use of it!

Here are the steps needed to debug a pre-built apk:
  • On the launch screen of Android Studio 3.0+, select the option "Profile or debug APK"
 

  • It will open up a dialog which will let you choose the APK you want to debug.
  • Ensure that the APK is build with debugging enabled
  • Next, Android Studio will try and create a new project in this folder ~/ApkProjects
  • Once it finishes loading the APK it will open up a screen that looks like this
  • As you can see it has unpacked the APK. It shows various part of the APK along with their sizes.
  • It has not fully decompiled *.dex files into *.java files. It show them as *.smali files. 
  • When you open the *.smali file, it will give you an opportunity to Attack Java Sources
 
  • Clicking the link Attach Java Sources will open up a dialog which will let you select the folder where Java sources are located.
  • Once you do this, you should see the Java classes in their full glory. You can now attach breakpoints and debug through the APK as if it was real source code.
  • If your project contains native code, it will let you attach a library containing debug symbols for that too.
  • Hitting the Debug or Run icons on the IDE will popup a dialog which lets you select the Device on which you want to install the APK and start debugging.

  • On selecting the device, IDE will install APK on the device and attach the debugger.
  • You should see a screen like this on the device

  • In a second or so, you should see your app's first screen loaded and ready to be debugged!
  • That's about all that is needed, you can debug, step through the code, evaluate variables and what not!
As you can see its a valuable little feature which can help in locating the bug in tricky situations!

Saturday, March 17, 2018

How to get to Tadoba and things to take along!

We had an opportunity to visit Tadoba Tiger Reserve, its a pristine and unique eco-system situated in the Chandrapur district of the Maharashtra, India. It contains some of the best of forest tracks and endowed with rich biodiversity. Its a brilliant place for nature lovers.

When I was planning the trip, one thing I found particularly challenging was to get pin point location of various safari gates and how to get there. This blog is an effort to help people like me, with landmarks to locate various safari gates. This post also list certain Do's and Dont's to best enjoy the safari.

Safari Gates and Getting There

Tadoba is close to a city called Chandrapur which is around 140KM from Nagpur. I drove to Chandrapur from Nagpur using a Zoomcar (Use my referral code MTI5NDU while making your first booking and get flat 15%(max discount - ₹1500) off).

Chandrapur has a bunch of decent hotels and is about 25-45 KM away from Tadoba Gates. Safari Gates are well spread out so make sure you are book a safari for a gate that is closer to where you are staying.

As a landmark locate Hotel Siddharth Chandrapur on Google Maps, the road to various gates of Tadoba is bang opposite this hotel.

We visited the following gates:
  • Devada Adegaon Agarzari Zone
  • Agarzari Zone
  • Moharli
  • Junona Zone
  • Zhari (Kolsa)




Devada Adegaon Agarzari Zone and Agarzari Zone are right opposite to each other. They are about 25KM from Hotel Siddharth.

Moharli and Junona are pretty close by as well and they are about 35KM from Hotel Siddharth. These gates are around 11KM from Agrazari Zone gates, basically you have to follow the same path but go a little ahead.

Zhari (Kolsa) is on the other side and is about 35KM from Hotel Siddharth.

Here is an interactive map to give you a better idea


Buffer Or Core Zone?

Tadoba has two types of zones, Buffer and Core. Some people say that, there is a high probability of spotting the tiger in core zone, but my personal experience is that its all about a little bit of luck and timing. You need to be at the right place, at the right time, to spot the tiger. To put things in perspective, out of the 4 buffer zone safaris we did, we spotted the tiger in 3 of them!

Safaris are done in open Jeeps with one guide and one driver. One safari typically last for around 3:30-4 hrs. You can book the safaris online from here.


How many safaris to do?

To have a decent chance of spotting the tiger, I would recommend doing at least 4-5 safaris. This gives you a higher chance of spotting the tiger. If you just do one or two safaris, there is a good chance that you would return without spotting the tiger.

We did a total of 5 safaris (4 in Buffer zone and 1 in the core zone). In one of the safari we were extremely lucky to witness a Gaur fight. The idea is not to keep looking only for the tiger, there are plenty of other animals which are equally majestic. Trust me, these safaris are well worth their time and money.

Some random clicks:
Spot the tiger!

I am watching you!
Do's and Dont's
  • Lot of dust will settle on you and your clothes while doing the safaris. Its advisable to carry a scarf or handkerchief to tie around your head and mouth.
  • It will be cold, especially during the morning safari. Do carry something to cover yourselves so that you do not feel too cold.
  • Take ample water to drink and keep yourselves hydrated.
  • Carry some light weight foods like sandwiches to eat if you feel hungry during the safari -- Especially if you have young children.
  • Food options near and around Tadoba Gates are pretty limited. There is a nice MDTC Resort nearby with a decent restaurant. 
  • Please don't litter -- Anywhere!
  • Be patient and don't get disappointed if you don't spot a tiger. There are lots of other animals in the forest to see too. Even the flora and fauna has a lot to offer!

What a hot afternoon!



Closing Remarks

Its a must visit place according to me. If you are a nature lover, you would love to visit Tadoba. Its a very well maintained and protected national park too!

Saturday, February 24, 2018

Using jsoup with Kotlin to parse HTML

Time and again, I have had a need to parse HTML using Java -- And I have hated it. Partly because of much better tools in other languages.

Recently, I had one such need. I needed to,
  • Fetch HTML response from a URL
  • Parse it and scrape information from it
  • Dump it somewhere
  • Do all of the above using Kotlin
Yuck! Parse HTML in today's world? Unfortunately, there  was no known public API that would return JSON or XML or something else. The information was only available as HTML and only way to get that information was to parse and scrape it.

With that in mind, I went and looked out for libraries available to parse HTML using Java or Kotlin. I stumbled upon jsoup.

Its a nice lightweight library to parse real-world HTML. jsoup API is more or less similar to jquery API -- Which makes it a pleasure to use. Without wasting much time lets just jump right into code.

How Do They Do It!

Lets say, we just had a simple requirement, parse the Google Search Result Page and list all the result title's and URL's.

NOTE: I know that google does expose search API to return JSON response, but for the sake of this example just assume it didn't have any such API.

jsoup can be included via many ways. Here's how we could include it via a gradle file.

Next up, lets write a simple test, it will do all of the above mentioned things. Here are the relevant parts of the code.

This prints out the search index, title and URL from the search result page. Here's the sample output

That's about it!

PS: Here's the link to the sample code used in this post.

Saturday, January 27, 2018

Integration Testing first RESTful web service with Kotlin and Spring Boot

We have been building a lot of infrastructure using Kotlin and Spring Boot -- And I am loving it! I have decided to start documenting parts of it for my own reference. I also hope my rant is useful to someone else, who's trying to solve similar problems :D

In this post we will be looking at building our very first RESTful web service using Kotlin and Spring Boot and writing an integration test for it. We will be using Gradle as our build tool.

Gradle File

First important component in the puzzle is the Gradle file. Following is the complete gradle file that we will be using for this post. Notice the use of all-open plugin. This is needed so that Spring can Autowire dependencies into our Kotlin classes.


Code

Lets create a very simple Book data class, it has got properties like title and author.


Adding a BookController, it has only one method which serves the requests from /books/search?title={title} path. This method will return the details of the books whose titles contain the searched term.

Note: The books collection is hardcoded for this example but nothing stops us from getting it from an external source, e.g. a database.


Next we need to add the application class which will expose the main method.


Test

The fun part about Spring Boot + Kotlin is the ease of testing this service. We will test this service out in two ways

  • Testing the API in browser
  • Writing a integration test 

Browser Test
  • Boot up the server using the following command
  • If everything goes great you should see a message informing that the server is up and running.
  • Open up the browser and open the http://localhost:8080/books/search?title=Steve
  • You should see the following response in the browser

Integration Test

Spring Boot provides a convenient class TestRestTemplate to test out the REST api's. Here is an example test for testing our API


As you run the test, notice that it boots up the server, invokes the api and fires the asserts on the responses!

Bonus

The toJson and fromJson methods are extension methods, they convert Objects into JSON representation and visa-a-versa. I use them all the time, they are pretty handy!

Project Structure

The best way to show this is via a screenshot.


Thats about it, we have a web service written in Kotlin using Spring Boot and we have successfully written a test for it as well!

PS: Here is the complete source code of the example used in this post


Friday, September 30, 2016

ICICI Bank and their goof-up's!

This is yet another post to highlight goof-up's made by ICICI Bank (one of the biggest banks in India).

What I wanted to get done?

My current debit card had a problem because of which it used to work in some ATM's and some ATM's refuse to accept it.  All I wanted, was to get a replacement debit card.

Goof-up 1
  • I went to the ICICI Bank branch which is almost next door to where I stay.  
  • Told them I want to get a replacement debit card
  • The bank representative took the request and assured that the card will be delivered in 10 days
  • To my surprise after a few days I received a message saying my new cheque book has been dispatched to my communication address.
  • So although I wanted a replacement debit card they decided to send me a cheque book instead :P
Goof-up 2
  • Since visiting the branch didn't seem to solve my issue, I decided to place the request for my replacement debit card using Phone Banking.
  • The lady took the request and gave me the tracking number as well.  
  • I checked with her, the address to which the card will be delivered and it was accurate
  • She assured me that the card will be delivered in 10 days.
  • After a few days, I got an SMS saying the card has been dispatched and I got the tracking details of the card as well.  The card was being dispatched from Delhi.
  • To my surprise, I found that the card was going to Ahmedabad!  
  • From Delhi why would the card go to Ahmedabad when I stay in Pune?
  • I called up the call centre to find out whats going on.  The lady said that, "from Ahmedabad it will come to Pune, don't worry".
  • To my horror when I tracked the card again (after a couple of days) it said it was delivered to someone in Ahmedabad!
  • I immediately called the call centre and they said its delivered to some ICICI Branch at Ahmedabad.  The lady said that she is marking an email and ensuring that the card gets delivered to my Pune address from Ahmedabad.
  • I was like WTF is going on!  
Goof-up 3
  • After a few days I realised that they have send the card from Ahmedabad to my old communication address @ Ankleshwar via Speed Post.
  • Obviously, I don't stay there right now, so I called the callcenter and told them about this error and asked them to recall the card immediately.
  • The lady again assured that the card won't be delivered since you are not staying at that place and will come back to one of the ICICI Bank branch.
  • This was getting tricker by the minute.  
  • After a few days another Surprise, the card was delivered to my old communication address to someone without even checking for ID Proof.
  • Wait wait there is more, even the debit card pin got delivered to the same place.
  • I was lucky that it was collected by my old neighbour but it could have very easily landed into hands of any un-trust worthy person.
  • By this time I was fed up with them and I decided to write to them about the whole episode and here is their response:
We inform you that as you are in non receipt of the debit card, you may call customer care and block the debit card through IVR and place the request for reissue of the debit card to be delivered to any of your nearest ICICI bank branch so that it gets delivered in 05 working days and you may collect it from the branch.
  •  They tell me to block the debit card number which was not delivered to me - Then how the hell would I know the number @#$@#$
  • They also asked to request a replacement debit card all over again!! - Its like, I have nothing else to do in life but to play Debit Card, Debit Card with ICICI Bank.
I fail to understand how such a big bank which is supposed to have state of the art software and facilities can operate like a bank from 1980's?

PS: All events mentioned in this post are completely real and I would recommend everyone dealing with ICICI Bank to be extremely careful with your account and its security.
Have some Fun!