The promise of the G Suite developer program is too difficult to ignore:
As they say: millions upon millions of G Suite users.
Moreover, there is Gmail, Google Sheets, Google Docs, Google Analytics, and a huge set of Services to choose from.
Or so I thought
I made three Apps (two web apps and one add-on):
(1) AutoReply for Gmail (web app), (2) Rapid MailMerge add-on for Google Sheets, and (3) Hustlr.io (web app).
(I made an attempt to release a Gmail add-on some time back and had to give up the attempt. That sob story can be read here.)
This article is my experience trying to get Google approval for my app Hustlr.io. I am posting this in the expectation that someone else trying out the same process may find it useful.
The Web App – Hustlr.io
The idea of Hustlr.io is simple. You have a spreadsheet with your contacts. You want to send emails to those contacts in your spreadsheet and collect responses from them. The responses gets collected in the same spreadsheet. Imagine you are organising an event. First, you get all your contacts in the spreadsheet and send the email. Then, you receive your RSVPs in the same spreadsheet. G Suite has both spreadsheet and Gmail, so a user with a Google account can log in and immediately start using the service.
For future versions, I had planned advanced messaging capabilities making use of the power and simplicity of the spreadsheet with custom routed messages.
For a taste of this, imagine you oversee a large event with a large number of attendees and multiple teams working on it. All the relevant information is in a single spreadsheet. How would you want to handle messaging at each step? Hustlr.io will be an answer for that.
Meanwhile, I had built Rapid Mail Merge – a Google Sheets add-on with similar but limited features. The limitation of a Google Sheets add-ons is that it is very much limited by the restrictions of Apps Script. It just can’t have the smarter features of Hustlr.io.
It took me around 4 months to build an MVP version of Hustlr.io. The next step is getting verified by Google. Since the app uses Google APIs, I must first get it approved. The Hustlr App uses the Google Sheets API to read and update Google Sheets. Gmail is one of the options for sending email. So, the App requires approval for Gmail API scopes as well.
While the app is under review, an attempt to log in using a Google account will throw an error like this:
29 Jan 2019 – Submitted Hustlr.io for verification
When I started making the app, the approval process was so simple and easy. I already had the Rapid MailMerge add-on successfully published in the G Suite marketplace. By the End of January, Google’s policies had changed. Any App that uses Gmail now has to go through a very tight approval process. I submitted Hustlr.io for verification on 29 Jan 2019.
22 Feb 2019 – Request Denied
Lesson: There shouldn’t be anything that indicates your App is not “ready”
It is common practice for Apps to go through several rounds of releases (alpha, beta, etc) It doesn’t mean that the app is not completely built. Releasing to a limited set of audience allows iterative improvements. I initially thought I will release Hustlr.io to a small set of users and get the feedback. This helps in fine-tuning the on the UI. The back-end of the app, of course, was ready. I didn’t want a casual lurker to signup just yet. The app is still not approved by Google anyway. So I had attached a small beta label to the header.
The approval process will immediately reject your application if there is any indication that your app is not “ready”. Owing to the beta label, I got rejected.
The only reason for concluding “development and staging” app is that I had a “beta” label in the title.
Remember, Gmail was in “beta” for several years!
I removed the beta label and submitted the request again. Then sent a reply to the email that the app is not in “staging” but ready for production.
22 Feb 2019 – Requires a video demonstrating the App and the scopes
Explain each scope and why the App requires the scope.
I have to explain OAuth scopes a little bit here. When you apply for the API access, you have to provide the list of “scopes” that the App will be using. A scope is like a group of API functions . If you are given access to one group, you have access to all the functions in that group. When Google shows the “permissions” screen to the user, the list of permissions screen is based on the scopes allowed to the App.
For example this scope:
https://www.googleapis.com/auth/Gmail.compose
Allows the app to compose and send emails on behalf of the user.
The problem with scope is that the Google API scopes are not granular enough.
For example, for Hustlr.io, I wanted to allow the user to search and find their Google Sheet. The app does not need access to anything else, just the Google sheets. However, the scope that gives access to the functionality I need is
https://www.googleapis.com/auth/drive.readonly
When the user is requested to review the permissions, It would ask:
“See and download your entire Google Drive”
This will cause confusion to the user because the App has nothing to do with his drive!
The App does not need access to the whole drive either.
Moreover, the app is requesting another scope which is
https://www.googleapis.com/auth/spreadsheets
Anyway. That scope allows, reading, writing deleting modifying spreadsheets.
Similarly, sending an email for the user (using the user’s Gmail account) sometimes requires the ability for the user to select his “from” address. In Gmail, you can configure more than one email addresses and you have the option to send email as any one of them. This is a very convenient feature for the user.
However, getting this setting requires scope access to the user’s whole settings. Bundling the access to the “Send As” with Gmail.compose scope would have avoided the need to ask for the whole Gmail settings.
The bottom line is that most of the time, the App will have to request for a broader scope where all that it needs would be a tiny piece of information.
So the next step in the approval process is to send a video demonstrating the use of each of the scopes.
I sent the response with the video link on 25th Feb 2019.
Then I waited and waited.
5th March 2019 – Follow-up
I had not received any response from the approval team. So I sent a follow-up email. No response either.
7th March 2019 – Twitter follow-up
Overwhelmed by the waiting, I sent a tweet addressing as many Google handles that I knew
Thankfully, @GCPCloud responded.
I sent a private message with all the details. @GCPCloud recommended that since I have a G Suite account, there is an option to email “API support department” from the G Suite Admin interface. Use that option to follow up with the support. So there comes the next lesson.
Use G Suite support to follow up Approval requests
One way I discovered to reach an actual human at Google is to use the G Suite Support system. Open G Suite Admin console and click on the “?” Then click “contact support” and select email ( I selected email. Phone and Chat may be possible ) While emailing select “G Suite API support”.
The advantage of using this system is that follow up is easier. The answers are quicker. I guess the matrices of the support system (response time, customer ratings for the support, etc) are measured and monitored.
A customer support representative responded almost immediately. I explained the problem. She said she will reach out to the team that handles the approval process. She did. I was in tears – first humane response from a human at Google!
I got an answer from the approval team the following day.
8 March 2019 – Agree to what you had Agreed to Agree
The next step was to agree to the terms of the restricted scopes and the requirements for the restricted scopes. Restricted scopes are those scopes that Google recently assessed to be very risky for them to share but shared anyway. At the moment this includes almost all parts of the Gmail API.
Essentially All I want is the ability to send email on behalf of the User. It is exactly the same as what Amazon SES, Sendgrid, SendInBlue and scores of other services offer. The advantage is that the user already has Gmail account and don’t have to be bothered with a complicated setup.
The requirements that Google puts forward in the “restricted scopes” is primarily not to share user information with any third party You have to declare this stand precisely in the Privacy policy.
Privacy Policy
Your privacy policy should clearly state that you will not share any of the user data with any third party.
Another App of mine (auto-reply for Gmail) which was going through re-verification got rejected because the Privacy policy said it shares user data with Google Analytics. The wording in the privacy policy was not correct. It doesn’t share any user data, even with Google Analytics. All it had was the analytics script to count the number of visitors. It didn’t even have goal tracking.
This also means that you may not be able to use third party services (like MailChimp or Drip) I was planning to use one of such service to follow-up with users on their experience with the App and do surveys.
I got the privacy policy amended like this to be compliant with the restricted scopes requirements
I came back to the email asking me to agree to the restricted scopes requirement, I replied “I agree “
The funny part is that I had agreed to those terms while submitting for the approval. The popup already mentions those requirements and you have to agree to those terms to submit the approval request.
So I agreed and waited.
15 March 2019 – “We are in progress”
On 15 March 2019 out of the blues I got a reply to the OAuth request that my application is in progress:
Surprisingly, I got the same email for all the re-verification requests as well (I had two other re-verification requests in parallel – one for Rapid MailMerge and another for Auto-Reply)
So it should be a mass email, maybe to pacify the developers.
28 March 2019 – “Compliance”
…
On 28 March, I got a response that the App was ready for security assessment by appointed security assessors.
The reason quoted is a little confusing. It says that the assessment is because the app is sending data to third-party servers. I guess this means that since the App is hosted on non-Google servers, I have to go through this assessment. The natural conclusion from this statement would be that if I host everything in Google Servers itself (App Engine, Cloud SQL) it would approve without this third-party security assessment.
I replied to the email asking this very question. Didn’t get any reply.
I then went through G Suite email support. This was their reply:
I concluded that the third-party security assessment is required regardless of whether the App sends data outside Google Servers or not.
The next step, then, is for Hustlr.io to get a “security assessment” done by a third-party assessor, costing $75000. We will see how that goes.
The idea of Hustlr.io is not “proven”. I mean, before I can release the App, I have to get approvals from Google. For that, I have to get “security assessments” done with a heavy fee.
What brought us here?
Beyond Facebook’s Cambridge Analytica data privacy scandal, there is heavy panic in the tech world regarding how/what user data gets shared with third parties. Articles like “Tech’s ‘Dirty Secret’: The App Developers Sifting Through Your Gmail” amplified that panic.
The panic left all the big companies wondering where the holes were and how to patch them without hurting themselves. One easy solution, and the one that makes lawmakers happy, is to apply a heavy bureaucracy to the mix. A bureaucratic solution (“yes, we are reviewing and policing”) would be easy to explain both to the press and to governments.
On the other hand, a technical solution would allow developers to ask for the exact scope that they would use. Then make those scopes as atomic as possible. This would require the APIs to be designed in a more precise manner.
Remember that in the Analytica case, they were able to access more than they were allowed to.
But, is a bureaucratic solution sustainable? Let’s imagine every API provider who follows the suite starts demanding a kind of “certification” from a third party. Then, every year, you’ll have to keep renewing each of these “certifications”. Imagine this in a rapidly moving tech space.
I have read that in Post-independence India, until around 1991, in order to start an industry producing anything, you had to get up to 80 or so licenses from various government departments. One had to struggle through a maze of bureaucracy before they could start making anything. How did such a system come into being? The lawmakers at that time feared that the country would fall again to its status as a colony if it didn’t make its socialist bureaucratic red tape so tight that it couldn’t move.
Let’s hope that these developments don’t lead to a utopia wherein every new product requires “certification” before even getting its first user.
I feel that the actual solution to the real user data privacy concerns will be a technical one.
Update – 2nd April, 2019
As I was writing this log of events, I was tweaking Hustlr.io. With a little bit of tweaking and some compromise on user experience, I should be able to get rid of the “Restricted” API scopes. (Gmail settings, gmail read, access to Google Drive) So I changed the scopes required to just spreadsheets and “Gmail send”. Then applied again on 29 March 2019. I got the approval on 2nd April 2019. The lesson is that restrict the scope to as small as possible. Avoid “Restricted” API scopes as much as possible.
