Thursday, 23 March 2017

Using Awesome Tables with Google Form Attachments

Google recently added an attachment upload function to Google GSuite customers.  This functionality has replaced some workarounds, including one I have blooged about before using Folders

Google creates a Drive folder for the attachments, and lists the attachments URLs in the response sheet, separated by commas.  This is great, but to display the information in a Google site or similar, it is most useful to use an Awesome Table to display the results.  To get this formatted to suit an Awesome Table is a bit fiddly however.   This post explains a way to get this set up

First - here's the demo of the completed table



To begin with I created a new tab which is the one that the Awesome table will read from, manually inputted the column names in the first row, added filters on the second, and in the third row added in a query to bring the data in from the form. 

As the file URLs are all in one column, separated by a comma and a space, it would not be possible to hyperlink these with a formula - the files need to be split out initially - this is done with an array formula with a split function, using ", " as the separator

Next a Google Apps script was added to list the URLs and names of the folder that was created automatically to contain the form attachments.  The code should be set on a trigger, on form submit.
The script formats the URL to match the same syntax that the Google form produces and will output to a tab I created called 'FileList'.

function search()  {
    var folder =  DriveApp.getFolderById("***INSERT YOUR GOOGLE DRIVE FOLDER ID HERE**");
  var files = folder.getFiles();
     var sheetName = 'FileList';  
var SheetIds = 2;
var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(sheetName);
 while (files.hasNext()) {
   var string = '';
   var file =;
 var data;
    data = [
      string + file.getId(),



End Result is as per the below screenshot when it runs

Now we have the URL and the Name, vlookups can be added to each of the 'Doc' columns to bring through the filenames

With all this done, the final step is to add a tab called template, which can then be used in the Awesome table setup.   Screenshot below, but the file is linked at the end of this post, in case you want to copy and paste.  The documents column takes the URLs and file names and hyperlinks each one with a generic document icon

In the Awesome Table setup - just reference the template to apply this

A copy of the sheet is here:

Hope this is of use!

Sunday, 12 March 2017

Marking assessments and generating certificates automatically via Google sheets, app script, forms, slides and Form Publisher

We recently had a requirement for users to complete a short assessment, to record which users passed, and for those who did, to receive a certificate.
To do this I used Google Apps script, Google Apps and a great add-on, Form Publisher

Here's a diagram of how it all works...

In the background collection sheet of the assessment quiz, formulae work out whether the answers are correct and score them a 1 or a 0 -  A formula at the end assigns either a 1 (pass) or a 0 (fail) based on the target score. Those who have achieved a pass mark, have their details pulled through into another tab, using a simple query.  e.g

=query('Form responses 1'!Y2:AE,"select Z,AA WHERE (Y=1)")

A similar tab can be created to pull through those who have failed just using Where =0 instead

How to then pass the details into a certificate is achieved using Google Apps Script.  This script populates a new form, which uses the Form Publisher add-on and submits it.

var sheetName = 'Mail Merge'; // enter the name of the sheet

function GenerateCertificate() {
  var CERTIFICATE_CREATED = "CERTIFICATE_CREATED";  //Define flag used to stop duplicates
  var form = FormApp.openById('abcdefghijklmnopq123456789');  //your form id goes here
  var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(sheetName);
  var rows = sheet.getDataRange();
  var numRows = rows.getNumRows();
  var values = rows.getValues();
   var data = sheet.getDataRange()  // Get all non-blank cells
   .getValues()     // Get array of values

      var startRow =1;  // First row of data to process 
   for (var i=0; i<data.length; i++){
    var formResponse = form.createResponse();
    var items = form.getItems();

    var row = data[i];
         var processedstatus = row[4];  // column - Column containing the flag to stop duplciates  (A=0)
      if (processedstatus != CERTIFICATE_CREATED) {     //If Flag text not found then.....
        //Take the first column data and input this into the next form  (e.g Full Name)
     var items = form.getItems(FormApp.ItemType.TEXT)
     var formItem = items[0].asTextItem();
    var response = formItem.createResponse(row[0]);     

         //Take the second column data and input this into the next form (e.g. Name of assessment)
    var formItem2 = items[1].asTextItem();  
    var response = formItem2.createResponse(row[1]);     

        //Take the third column data and input this into the next form  (e.g. date completed)
    var formItem3 = items[2].asTextItem();
    var response = formItem3.createResponse(row[2]);     

        //submit the form
        //set the flag to stop duplicates
      sheet.getRange(startRow + i, 5).setValue(CERTIFICATE_CREATED);



The script will only process those rows without the flag "CERTIFICATE_CREATED" in column E, once it has processed that row, it adds that text, to prevent duplicates being created.

In the background, when the script populates the user name, assessment and date into the next form and submits it.  Form Publisher merges the fields into a certificate template I set up in Google Slides, saves the certificate as a pdf, and then emails it to the person who took the test.

A useful feature of using a second form, processed by a script, is that sheets for different assessments / courses can use that same template and same 'certificate generator' form to create appropriate certificates.  All that is needed is the script in a background collector sheet to populate that form with the required data.

Monday, 28 November 2016

IMPORTHTML and Pivot queries in Google Sheets

{NB A copy of the sheet with all the formula used in this demo can be found here]

The National Institute of Health Research (NIHR) publish an annual table of national research activity.   In this demo, I am going to use this dataset in Google sheets, by extracting a subset of the information from the website with a formula, and then using a few different pivot formula to manipulate the data.

The table is published on the webpage here


To extract just the information for NHS organisations in Kent, Surrey and Sussex, in my Google sheet I have used the formula

=query(importhtml("", "table",1),"SELECT Col1, Col2, Col3,Col4,Col5,Col6,Col7,Col8 where Col3 ='Kent, Surrey and Sussex'")

Simple pivot

To pivot the results, you can use a pivot table in Google sheets, via the data menu, However you can also use a formula to generate a pivot

Here's a simple example (where the raw data is on a sheet called "Raw Data 1" - showing the number of recruits by organisation type.

=query('Raw Data 1'!A:H, "select sum(H) pivot B")

Transpose a pivot

To change the orientation, so that it looks more like a normal pivot table, you can use 'transpose' in the formula

=transpose(query('Raw Data 1'!A:H, "select sum(H) pivot B"))

Filter a pivot

You can also apply a filter parameter in your query, so for example the following shows the sum of recruitment for just Acute trusts

=transpose(query('Raw Data 1'!A:H, "select sum(H) where B ='Acute' pivot A "))

Use multiple columns in a pivot

You can also bring other columns into a pivot, as long as these are included in the 'group by' or wrapped in an aggregate function.   I have used a 'where' clause too to prevent blank rows being included in the pivot

=query('Raw Data 1'!A:H, "select A,sum(H) where G >0 group by A pivot B")

Sorting a pivot

Sorting a pivot is more tricky,  in the example below, I have used a query, to query the query ;-)

=QUERY( TRANSPOSE((query('Raw Data 1'!A:H, "select sum(H) where B ='Acute' pivot A"))) , "Select Col1,Col2 Order by Col2 desc ")

Tuesday, 24 November 2015

Adding comments to an Awesome Table via a toggle sidebar

This solution came about from a requirement we had for colleagues to easily add updates and comments to a monthly report which was itself refreshed each month with new data. As the data changed in the report it was necessary to find a way to get the existing comments to be kept and to show against the appropriate matching lines in the new data set. To do this, I used an Awesome Table, with a sidebar, linked to a form which could be toggled to show and hide. The form has a few fields which are pre-filled depending on the line of data, so that the comments then get matched up and shown in the table.

A demo of the solution is below, using some dummy data  (a copy of the spreadsheet can be found at the bottom of the post)

To start I created a Google Sheet with the data in and a Google form to collect the comments

To get a unique value per row that can match against the appropriate comment, I concatenated the first three columns of my data  (Trust, site and study ID).  The formula puts the title of the column in the first cell, 'Nofilter' in the second cell (used to define the Awesome Table filter) and in the remaining rows with data, the concatenation:


with a similar formula on the form result sheet, any matching results can be brought in with a query, but with a join function, to combine multiple matching entries, sorting the matching comments by most recent.

=join("",query(Comments!A$2:H,"select G Where A = """&A3&""" order by B desc"))

(NB - one annoyance is that the query cannot be combined with an array formula, so this formula is filled down in a standard way. If the data is submitted via a form as well, then an Importrange or a query function can always be used to bring in that data to another sheet which would then be used for the remaining calcs.)

The returned column combines the date stamp, formatted to show just the date, with the comment, and some HTML line breaks so that this displays well in the Awesome Table, which was obtained by the following formula.

=ArrayFormula(TEXT(B2:B," dd/mm/yy - ")&F2:F&"<br/><br/>")

Awesome Tables now use Templates  which is a fantastic new feature.  In the template I added
what would appear in the sidebar, which is an iframe of the form, prefilling three of the fields using the data in the rows.

<iframe Height="610px" src="${"Study ID"}&entry.1197506384=${"Site Name"}&entry.1805865420=${"Trust Name"}"></iframe>

I then added some JavaScript to the template so that when a button is clicked, the sidebar is shown or hidden, with the table width expanded or reduced accordingly.

function changeClass()    {  
document.getElementById("sidebar").className = "sideactive";
document.getElementById("parentChart1").className = "mainactive"; 
document.getElementById("parentChart1").setAttribute('style', 'width: 74%!important;height 1200px!important'); 
function changeClassBack(){
document.getElementById("sidebar").className = "sideinactive";
document.getElementById("parentChart1").className = "maininactive"; 
document.getElementById("parentChart1").setAttribute('style', 'width: 100%!important'); 

The spreadsheet used for this demo can be found here
Please feel free to take a copy.

Friday, 10 July 2015

Folders as a workaround for Google form attachments

Update December 2015 - for enhancements such as automatically sharing folders with recipients and a link directly to the folder view rather than the 'open in drive' view, please see responses to comments below this post

Google forms are really useful, but a limitation is that it is not possible to attach a document to a record  (as you can do with a SharePoint list for example)
      This script aims to give a workaround for that - when a user submits a form, the form entry is assigned a unique reference,  a Drive folder is created with that reference, and they receive an email containing a link to that folder to upload related documentation.  Using an Awesome table, the link to the folder as well as the details in the form can be displayed together to make it easy to match documents to form items.

To being with, I created a Google form, and a Google sheet to collect the information.
To create a unique reference I used the following formula in the first row of the first empty column (where a username is in column B)  The formula puts the title "Reference" in the first row, in the second row it puts "NoFilter" (for use in the Awesome Table) and for any subsequent lines, it takes the first letter of the users first name, two letters from the surname, and the row number to make the unique reference

=ArrayFormula(IF(ROW(A:A)=1,"Reference",IF(ROW(A:A)=2,"NoFilter",((LEFT(B1:B)&MID(B1:B,FIND("#",SUBSTITUTE(B1:B&" "," ","#",1))+1,2)&MID(B1:B,FIND("#",SUBSTITUTE(B1:B&" "," ","#",2))+1,1)&ROW( A:A ))))))

Scripts are here:
var formURL = ''; // enter the edit url for the form
var sheetName = 'Form responses 1'; // enter the name of the sheet
var columnIndex = 8;  //the number here is a count with A = 1, it indicates where the link should go

// script to get the edit link

function getEditResponseUrls(){
  var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(sheetName);
  var data = sheet.getDataRange().getValues();
  var form = FormApp.openByUrl(formURL);
  for(var i = 2; i < data.length; i++) {
    if(data[i][0] != '' && (data[i][columnIndex-1] == '' || !data[i][columnIndex-1])) {
      var timestamp = data[i][0];
      var formSubmitted = form.getResponses(timestamp);
      if(formSubmitted.length < 1) continue;
      var editResponseUrl = formSubmitted[0].getEditResponseUrl();
      sheet.getRange(i+1, columnIndex).setValue(editResponseUrl); 

// script to create a folder for document uploads

function createFolder () {


  // Get current spreadsheet
  var ss = SpreadsheetApp.getActiveSpreadsheet();
  var sheet = ss.getSheetByName("Form responses 1");
  var data = ss.getDataRange()  // Get all non-blank cells
               .getValues()     // Get array of values
               .splice(2);      // Remove header lines

  // Define column number unique reference. Array starts at 0.
  var CLASS = 6;  //column G

// Get the ID of the root folder, where the sub folders will be created
  var DriveFolder = DriveApp.getFolderById("0123456789101112131415161718abcdefghijklmnop");

  // For each email address (row in a spreadsheet), create a folder,
  // name it with the data from the Class
  for (var i=0; i<data.length; i++){

    var startRow = 3;  // First row of data to process 
    var row = data[i];
    var class = data[i][CLASS];
    var folderName = class;
    var supplier = row[3];  //column D
    var folderstatus = row[9];  // column J
   if (folderstatus != FOLDER_CREATED) {
  //create the folder
  var folderpath = DriveFolder.createFolder(folderName).getUrl();

//send an email

  var emailAddress = row[2];  // Column C)
  var message = 'Thank you for submitting a purchase order requisition for supplier: '+ supplier +' - please upload the purchase order(s) relating to this to the following folder ' +folderpath ;   

      var subject = "Please upload related documentation";    
     MailApp.sendEmail(emailAddress, subject, message, {
  //  cc: ''         // optional cc
      //set the flag to avoid duplicates
      sheet.getRange(startRow + i, 10).setValue(FOLDER_CREATED);
      sheet.getRange(startRow + i, 11).setValue(folderpath);
      // Make sure the cell is updated right away 

Both scripts should be set to trigger when the form submits.
When a user enters information in the form, the scripts will generate an edit link to the item, and create a Drive sub folder with the unqiue ID. They then will get an email which contains a link to the newly created folder...

On a Google site, the results can be displayed in an Awesome Table...

 The Awesome Table setup used is below:

One quick note, as the unique ID is generated from the row number, deleting a row in the sheet would cause a mismatch between the references and any already created folders...
In my example above in the 'Advanced parameters', the Awesome Table is set to only show rows where A is not blank, so the method for removing any unwanted records from displaying in the Awesome table should just be a matter of deleting the timestamp for the unwanted row in the sheet.

Hope this is of interest. A copy of the demo sheet, including scripts can be found here

Tuesday, 7 July 2015

Enhancing Google forms via a customised webpage

Google forms have some great strengths, particularly in the way that Google Apps Scripts can work with the collected data. There's not much you can do though to the out of the box Google form to add functionality such as comparing fields for validation, adding word counts or any other JavaScript tricks. A way of getting the best of both worlds is to extract the Google form and put it into a web page, which can be hosted on Google Drive, and then enhance the design and functionality as you like with HTML and JavaScript.

The below video shows a few of the enhancements - including
-automatic word counts as you type in paragraph text fields  
-email address and tickbox validation to prevent form being submitted unless boxes are ticked
-accordion dropdowns to reduce form size
-working hyperlinks in the text. 

While these features are in the front end, in the back, as the fields are all still Google form fields, there is functionality in the collecting Google sheet so that when the form is submitted, Google Apps Scripts run, to provide an edit link, emailing the recipient with confirmation of the data and a link to edit, and a set timer so that the recipient is contacted every x months to review their data and amend if necessary. As normal with Google sheets, an add-on can alert the owners of the sheet when a form is submitted, and Awesome tables can summarise the data and provide searching and filtering.

Some of the code snippets are below:

I included a reference to jquery in the <head> for the accordion and field validation scripts

<script src=""></script>

If you have fields set to required in the Google form, if you try to submit the webpage version without completing these fields, then you are taken to the original Google form to complete these. This clearly isn't ideal, so its necessary to make the fields required instead using JavaScript
Sample code here for one of the fields

<script type="text/javascript">
// validate the First Name field
$('form').submit(function () {
    // Get the Login Name value and trim it
    var name2 = $.trim($('#entry_12345678').val());
    // Check if empty of not
    if (name2  === '') {
    $('#errorwarning2').text("Required field 'First Name' is empty, please correct and re-submit.");
        return false;

To do a word count, I used the following script
<script type="text/javascript">
function wordCount( val ){
    return {
        charactersNoSpaces : val.replace(/\s+/g, '').length,
        characters         : val.length,
        words              : val.match(/\S+/g).length,
        lines              : val.split(/\r*\n/).length
var $div1 = $('#count1');

$('#entry_987654321').on('input', function(){
var a = wordCount( this.value );
"<br>Word count: "+ a.words

The Div that shows the word count can then be put next to the relevant field
To stop the form submitting if the email address fields do not match, I used the following:

<script type="text/javascript">

// form id value, default is ss-form
var formID = 'ss-form';
var formKey = 'abcdefghijklmnop123456789';
var submitted = false;

$(document).ready(function () {
    var ssForm = $('#' + formID);

    ssForm.submit(function (evt) {
         var email = document.getElementById("entry_123456789").value;
        var confemail = document.getElementById("entry_1011121314").value;
if(email == confemail) {
            ssForm.attr({'action' : '' + formKey + '/formResponse'}); 
            return true;
        } else {
    $('#errorwarning').text("Error - email addresses do not match, please correct and re-submit.")
            return false;

If the user has mistyped an email address and has been unable to submit the form, I wanted the error message relating to this to hide as soon as they start typing in the email field again. For this I just used this:

$('#entry_123456789').on('input', function(){

Away from JavaScript  - in the Google Sheet, to let users update their details every x months, I used a variant of the script in my blogpost here

Hope this is of interest, any comments or queries welcomed

Tuesday, 23 June 2015

Using Metadata in Drive descriptions and display these using an Awesome Table

Update 19/10/2015 - Due to Google's update to their security sandbox - the code below has been updated to work with the new IFRAME method.  

One of the things I liked particularly about SharePoint was when you uploaded a document to a document library, you could add any number of custom fields for metadata, making displaying specific sets of documents easy on a site.

With Google Drive, although the search has its uses, I wanted to recreate if possible a way of adding metadata to documents on upload, and be able to display just certain documents on a site using this. The method I have used is a script deployed as a web app, which gives a user interface to upload the files to a folder, with a Google sheet to periodically list the contents of that folder, for displaying in an Awesome Table.

The first bit is to create a picker, to select files and add metadata. The end result will look like this

When a user uploads a file, they then get a confirmation of upload and a link to the file as per the below. The metadata goes in the description field of the file in Google Drive

For this I created a script and deployed as a web app

The first bit goes in -  I have included some sample metadata categories, but it is easy to change these as required.

/* The script is deployed as a web app and renders the form */
function doGet(e) {
  return HtmlService.createHtmlOutputFromFile('DrivePickerHTML.html')
/* This function will process the submitted form */
function uploadFiles(form) {
  try {

    var folder = DriveApp.getFolderById("abcdefghijklmnop12345");
    /* Get the file uploaded though the form as a blob */
    var blob1 = form.myFile;    
    var file1 = folder.createFile(blob1);    
    /* Set the file description as the name of the uploader */
    file1.setDescription(form.Division + "," + form.DocType);

          var file1url = file1.getUrl();
    /* Return the download URL of the file once its on Google Drive */
    return "File Uploaded - <a href='" + file1url +"' target='_blank'>click here to see on Drive</a>"
  } catch (error) {
    /* If there's an error, show the error message */
    return error.toString();

The next bit goes in a HTML file which I have named "DrivePickerHTML.html

Division / Specialty Document Type Choose a File Upload
Next in a Google sheet, I added a script to list all the files in a folder.  This creates the necessary headers and filters for the Awesome Table.  The script then lists all the files and details, including the contents of the description field, which now contains the metadata. A trigger can be set to refresh at whatever frequency is required.   As part of this script, it also checks to see if a file has been added within the last few days, and if so, gives it a "!New" icon.

var ss = SpreadsheetApp.getActiveSpreadsheet();

function GetFolders(){
var ss = SpreadsheetApp.getActiveSpreadsheet();

     var sheet = ss.getSheetByName("Sheet1");
  sheet.appendRow(["Icon", "Name", "Date Modified", "Size", "Category", "File / Folder"]);
  sheet.appendRow(["NoFilter", "StringFilter", "DateFilter", "NoFilter", "StringFilter", "CategoryFilter"]);
  var mainfolder = DriveApp.getFolderById("abcdefghijklmonop12345"); //change this bit to your folder ID

 var contents = mainfolder.getFolders();

var file, data, rows = [];
  for (var i = 0; i < contents.length; i++) {
    file = contents[i];

 data = [ 
       "<a href=" +'"'+ file.getUrl() +'"'+ " target='_blank'><img src=''</img></a>",
      "<a href=" +'"'+ file.getUrl() +'"'+ " target='_blank'>" + file.getName() + "</a> ",


function listFilesInFolder() {
  var folder = DriveApp.getFolderById("abcdefghijklmnop12345"); //change this bit to your folder ID
  var contents = folder.getFiles();
     var sheet = ss.getSheetByName("Sheet1");
  var file, data, rows = [];
 while (contents.hasNext()) {
   var file =;
       var filetype = "application/icon_10_generic_list";

   var mime = file.getMimeType();   
      if (file.getBlob().getContentType() == "application/pdf") {
      filetype = "application/icon_10_pdf_list";
        if (file.getMimeType() == "application/") {
      var filetype = "application/icon_11_spreadsheet_list";
         if (file.getMimeType() == "application/") {
       filetype = "application/icon_11_presentation_list";
       if (file.getMimeType() == "application/") {
       filetype = "application/icon_11_document_list";
        if (file.getBlob().getContentType() == "application/vnd.openxmlformats-officedocument.presentationml.presentation") {
       filetype = "application/icon_10_powerpoint_list";
              if (file.getBlob().getContentType() == "application/vnd.openxmlformats-officedocument.wordprocessingml.document") {
       filetype = "application/icon_10_word_list";
          if (file.getBlob().getContentType() == "application/msword") {
       filetype = "application/icon_10_word_list";
            if (file.getBlob().getContentType() == "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet") {
       filetype = "application/icon_10_excel_list";
             if (file.getBlob().getContentType() == "application/") {
      filetype = "application/icon_10_excel_list";
        if (file.getBlob().getContentType() == "application/") {
       filetype = "application/icon_10_powerpoint_list";
      if (file.getBlob().getContentType() == "application/") {
      filetype = "application/icon_10_powerpoint_list";
      if (file.getBlob().getContentType() == "image/png") {
      filetype = "application/icon_11_image_list";
     if (file.getBlob().getContentType() == "image/jpeg") {
      filetype = "application/icon_11_image_list";
     if (file.getBlob().getContentType() == "image/jpg") {
      filetype = "application/icon_11_image_list";
     if (file.getBlob().getContentType() == "image/gif") {
      filetype = "application/icon_11_image_list";
      if (file.getBlob().getContentType() == "video/x-msvideo") {
      filetype = "application/icon_11_video_list";
      if (file.getBlob().getContentType() == "video/webm") {
      filetype = "application/icon_11_video_list";
      if (file.getBlob().getContentType() == "text/css") {
      filetype = "application/icon_10_generic_list";
     if (file.getBlob().getContentType() == "application/x-javascript") {
      filetype = "application/icon_10_generic_list";
     if (file.getBlob().getContentType() == "application/xml") {
      filetype = "application/icon_10_generic_list";
   Logger.log(mime + ":" + file + ":" + filetype);
    var nowdate = new Date();
     var nowtime = nowdate.getTime();
     var createddate = file.getLastUpdated();
     var createdtime = createddate.getTime();
     var timediff = nowtime-createdtime
     var differencecompare = timediff/1000/60/60/24
    var newicon ="";
    if(differencecompare < 3) {
     var newicon = "<img src=''</img>";
    data = [ 
      "<a href=" +'"'+ file.getUrl() +'"'+ " target='_blank'><img src=" +'"'+ "" + filetype.substring( filetype.indexOf('/') + 1 ) + ".png" +'"'+"</img></a>",  
      "<a href=" +'"'+ file.getUrl() +'"'+ " target='_blank'>" + file.getName() + "</a> "+newicon,


Finally, the Awesome Table can just be created to read from this sheet, allowing documents to either by manually filtered or searched for by metadata, or hardcoded in the Awesome Table to just show documents containing specific metadata

Hope this is of interest, any questions or comments welcomed