How to lead a productive Proof of Concept

Review some tips and tricks from a Solutions Engineer on how to successfully manage a Proof of Concept with your customer.

Welder spark - Photo by Rob Lambert

In my previous article, I have discussed some Sales Engineering tips about the qualification and scoping phases in a pre-sales engagement. One of the areas I briefly covered was the Proof of Concept (PoC). I think this is a topic worth expanding on - something we will unpack in this article!

( Photo by Rob Lambert )

PoC fatigue is real

Here lies a Solution Engineer. They fought valiantly and juggled many competing requests, responded to countless e-mails and advised on twice as many technical questions (I'm leaving to the reader to calculate "twice" of "countless").

Alas, they ultimately succumbed under a concerted barrage of Customer objections and super-urgent asks from Executives and gave up on this worthy endeavour.

They now live a secluded life away from the tech world and enjoy their days in a small village, tucked away on the coast of Cinque Terre.

Vernazza - Cinque Terre ( Photo by Kirsten Velghe )
Vernazza - Cinque Terre ( Photo by Kirsten Velghe )

Who wouldn't want to partake daily of such glorious scenery, you say? What if I told you that instead of giving up you could remote work from one of the houses in the picture and happily combine your duties as a Solution Engineer with that dream seascape? Did I pique your interest? Continue reading then!

What is a Proof of Concept

In the world of Pre-Sales, a Proof of Concept  is an optional stage of the Technical Qualification and Scoping process. It allows a Customer to check and validate that a product, feature or service being evaluated for purchase will meet (or even exceed) the needs - concerning specific Customer requirements.

It is optional because many times our Customer is happy with our initial explanations and clarifications. Maybe this Customer already used the same product/service at a different company, and they know it even better than you do! In other cases, the requirements are so straightforward that the feasibility is already fully determined by reading your product documentation.

For a Customer, a PoC can be free or paid. It depends on the policies of the organization that is offering it. Ultimately, a PoC always has a cost: perhaps it may not cost money to initiate one. However, the people agreeing, designing and running the tests usually are in receipt of a salary.

From physics, we also know that the time a person spends on that Proof of Concept is non-refundable under any circumstance.

A PoC is a scoped activity. Above, we mentioned "specific requirements": in other words, there should be a defined Customer situation reflecting the Customer needs and setting the scene for the tests to be carried out. This is important to mention because sometimes a PoC can be confused with similar, non-scoped activities:

  • A Customer could be applying for a "Trial" period on a set of products or features. Typically, this gives them full, unsupported access to the features for a fixed period.
  • In other cases, a Customer may sign up for a "Pilot". This is a more formal evaluation period, generally underpinned by an executed commercial agreement. The "Pilot" can also have a scope but is generally much broader / high level than a focused PoC.

In a Proof of Concept, we can define specific Success Criteria and Measurements. We work with our Customer and together agree on these aspects by refining the overall high-level requirements. The purpose of the PoC is for our Customer to determine if these criteria are met and consequently if our proposed Solution would meet the requirements.

Now that we have a shared understanding of what a PoC is, let's look at the lifecycle of a PoC and let's distil some best practices for each.


Qualify the PoC

The first step is to determine whether a PoC is the appropriate next step for the engagement we are currently involved in.

In an ideal case, we will find ourselves in one of these two scenarios:

  • Following initial discussions/demos, the Customer has expressed interest in validating specific features and use cases related to one or more products offered by our organization. As these scenarios directly contribute to the overall value proposition of purchasing the Solution, the Customer is unwilling to move forward unless these scenarios have been validated from a technical standpoint.
  • Similarly to before, we discussed the requirements with our Customer. During the qualification phase you, the SE, have identified uncommon or novel usage scenarios for some of the products being evaluated. Additionally, these scenarios have been flagged as "must-have" capabilities by the Customer. Therefore, you suggest a formal technical test phase to guarantee that we can meet the requirements.

If you look closely, in both scenarios the need for a Proof of Concept is clear: either the Customer or yourself have identified crucial aspects to be validated for the Solution to make sense.

In the second scenario you, as the trusted advisor of the Customer, are proactively identifying an area requiring technical proof: the Customer relies on your experience and your knowledge of the product - so you can validate the Solution together. The Account Executive (AE) partnering with you can also highlight that, by being forthcoming and solution-oriented, we are showing the Customer that we can be trusted.

After all, we could just ignore that pesky scenario, move ahead and conclude the agreement, couldn't we? And then it works - if we are lucky. Luck eventually turns and suddenly a corner case destroys our house of cards. As Walter Sobchak eloquently put it in a great movie: "You're entering a world of pain" (and you haven't done your job properly either...).

Just to make it clear, we also shouldn't offer a PoC to validate a scenario that we know works for sure, or that is spelt out line by line in the product documentation. That is what existing Customer Success Stories are for. Remember that the PoC is a time/resources investment for all parties, so it shouldn't be offered without a reason, and not before we have attempted to explain in detail the common scenario we have in front of us.

Before moving to the execution phase of the PoC, work with your AE to ensure the engagement is well qualified: for example, does the Customer know at least a ballpark cost for the solution, and do they find it acceptable? Do they have the time and resources to support it to the end? Nothing worse than completing successfully all the tests, only to discover the Customer didn't have the budget for the Solution in the first place!

Here is another scenario that can happen: the AE proposes running a PoC too early in the qualification phase so that the Customer can "look around at the products". There is a risk to waste everyone's time here, due to the lack of focus and the impossibility of making this a targeted, time-bounded effort. In my opinion, an unsupported Trial Period (if offered by the organization) would be more appropriate here.


Puzzle pieces - Photo by Mel Poole
Photo by Mel Poole

Agree and Scope a PoC

If we determined that a PoC is a good idea, then the next step is to translate the "good idea" into an actionable plan. This is done by working with the Customer to establish what is going to be tested, for long and by whom.

In my previous article, I spent some time on this point and argued that the SE should take the lead here and guide the Customer by creating a PoC execution plan (or PoC document). I'll include again the list of elements that should be part of this document:

  • Key stakeholders on all sides (including partners, if involved).
  • Key customer dates (both for the POC itself and for the solution deployment).
  • The products that are part of the POC.
  • The details of the environments to be used for the tests.
  • Scenarios & Success Metrics:  more on that later.
  • Assumptions & Limitations: more on that later.
  • Timeline: how long the POC will run? When are we meeting to check progress, and to draw conclusions? By which date do we need a Technical Proof Outcome decision? What happens when the POC is deactivated?
  • References to our documentation: for easy access during the tests and later.

As I said earlier in the other article, in my experience Customers appreciate when a structure is brought in this process. Another great advantage is that the document is yet another occasion to confirm a shared understanding between you and the Customer. Did you misunderstand a scenario? Did the Customer miss a key aspect on one of the features that you have demoed?

Isn't it great that you have a moment to validate before starting, and not have those misunderstandings creep up unexpectedly on you while everyone is sprinting to complete the tests with their blinders on?

Finally, whether the PoC is paid for or not, the document itself  can be thought of as a proper deliverable. It is shared with the Customer, and let them see first-hand what it feels like to be working with your Organization. They can reference it in later phases of the project for convenience.

Do you really want to handle this in a chaotic, "e-mail thread from Hell" which crashes the Customer's inbox when opened, or have a collaborative document that can be bookmarked for easy access and reference?

Scenarios and Success Metrics

The core of your document is a table summarising the Scenarios to be tested, and defining what good looks like. I like to capture the following points for each scenario of interest:

  • Name: a one-liner describing the desired Customer outcome.  For example: "Reduce complexity in the Customer's order management web interface" or "Implement Apple Pay in the Customer's mobile checkout flow". Sometimes it also makes sense to list "non-functional" scenarios which are heavily influenced by the technical execution. For example: "Create a draft implementation plan for the features to be deployed" could help a Customer who wants to get a handle on what the rollout activities would look like.
  • Product / Feature: here we want to tie the scenario to the specific feature(s) to be deployed/implemented to solve it. This helps link the successful technical outcome to the overall value proposition for the features included in the proposal.
  • Success Criteria: a bullet list of self-contained test steps that the customer will follow to verify the scenario. Here, we can provide bookmarks and links to our documentation, to make the task easier and indicate the blueprint for the future, real implementation.
  • Measurements / Metrics: each step in the Success Criteria must be tied to a measurable outcome. This can be a simple yes/no answer: "When pressing checkout, the mobile app displays the Apple Pay option to the user" or "The administrator can issue an order refund from the order details page in one click" ). It can also be a measured quantity. Measurements can be tricky, especially when we want to compare to existing or different solutions. In such cases, it is important to anchor them to a reference scenario: for example "Cumulative Layout Shift on the test page is improved by 90% - compared to the reference value XYZ".
  • Outcome: this will be a blank section at the start of the process. We will track the results for each test step as we progress. We will also note down if the objective was fully, partly or not met, and who/when made that determination on the Customer side.

The good news is that you can help your Customer draft this and reduce the effort required to get going: based on the information you have gathered during the qualification phase, you should have a good grasp of the key scenarios, and propose criteria and measurements. Your Customer can then review your draft and add/remove/amend as needed. This is much better than giving someone an empty table and asking them to fill it in by the next day.

Don't forget to document in writing any fundamental assumption that is being made, either by yourself or by the Customer. If you can eliminate such assumptions in the first place, even better. If not, at least you have a shared list in front of everyone, which you can call attention to when appropriate. Keep it in mind, and aim to validate these assumptions during the PoC where possible - as this dispels possible areas of doubt about your proposed Solutions.


Initiate and Support a PoC

Having covered the crucial setup steps, both you and your Customer are already well underway to achieve a successful outcome. What will tip the balance of the scale one way or another will then depend chiefly on these three aspects:

  • Technical Fit
  • Communication
  • Accountability

Let's see in more detail what I mean here.

Technical Fit

This is the easiest to understand. With an agreed PoC scope, underpinned solidly by the documented criteria and measurements, it is time to demonstrate that these can be indeed achieved successfully.

This is where your technical prowess will come to shine, enabling you to coach and educate your Customer about the qualities of your Solution. You will be helping them throughout by providing relevant documentation, being available in tackling questions and in helping if issues are encountered.

Depending on how your organization operates, and the type of product or service your Solution is made of, you could also get involved in creating draft prototypes/code templates to be handed over to your Customer for adaptation.

Communication

The power of consistent, clear communication throughout the PoC process should never be underestimated. With the help of your AE, reach out to the Customer and clearly state by when the PoC document needs to be completed, to begin the tests by the agreed date.

Here, the Timeline section of the document will help you: if you know when the Customer wants to begin, create an indicative execution plan, starting from the day of the kick-off call. This will be a simple list of target dates, actual dates, the task to do and who is responsible for carrying it out. Keep track of the actual date of completion for each, to highlight any deviation from the original schedule. If this happens, find out why - for example:

  • Is it taking longer because the Customer is having a hard time testing a specific feature? Well, you need to jump in feet first on that one and get to the bottom of it.
  • Is the Customer team being diverted away from the PoC? Maybe there is an unexpected, higher priority task that needs attending. Or perhaps there is less commitment to progress? Maybe there are more fundamental doubts about the Solution? Work with your AE to understand what is going on.

You will frequently adapt your style of communication to what is preferred by your Customer. If they prefer e-mails, check in regularly and summarise where we are in the process and what are the next immediate steps to progress.

Do they prefer to work with shared documents, tagging people with comments and handling it in one place? Happy days, I love it when that happens!

Have the communication lines crumbled, and you cannot get any answers from the Customer? This is a red flag, and it should be flagged as soon as possible to your AE to be addressed.

Accountability

This is a quick one. On all sides, everyone should feel accountable for the commitment that has been made by the parties to execute that specific Proof of Concept in the agreed time.

You should block your calendar as appropriate to support your Customer for the duration of the PoC. You may need to jump on a call at short notice or respond quicker than usual for time-sensitive queries.

Equally, your Customer should show accountability on the task. The test is being done to their benefit because they want to prove that your proposed Solution will help them. They should take the time to conduct the agreed tests and keep you informed of the progress.

If things are not progressing as expected, give the benefit of the doubt. There may be genuine, unexpected issues on the Customer side preventing smooth progress. As always, team up with your AE to check what is going on and how you could help.

Is the Customer requesting an extension to the end date of the PoC? The rules here will vary across organizations, but in general, I believe common sense should be applied:

  • Is it taking longer because there was an issue on your side to enable the test environments? An extension should definitely be granted here.
  • Has the scope slightly changed and is there an interest in covering two more, well-defined scenarios? This is also a good reason in my opinion for extending a PoC.
  • Was someone unavailable to run the tests on the Customer side in the allotted time? To be considered. If this is a repeated circumstance and we seem unable to gain traction every time, then this is definitely a red flag.

Woman with umbrella - Photo by Edu Lauton
Photo by Edu Lauton

Review and Conclude the PoC

The last step of the process is arguably the most important. There is nothing more frustrating than getting to completion, only to discover that there is a disconnect somewhere in the conversation and the whole endeavour has now become a bit of a moot point.

If you kept tabs on the communication during the PoC, and also have teamed up with your AE to have knowledge of the overall discussions that are unfolding with the Customer, then the risk of this happening will be greatly reduced.

At a certain point, we will note that we have defined outcomes on most scenarios and we are getting close to the agreed end date. This is the perfect time to review the planned timelines and arrange a PoC wrap-up meeting with the Customer and the AE.

The agenda of the meeting will focus on reviewing the document, in particular the Scenarios & Success Criteria. We should be clear in our agenda by detailing that the meeting marks the conclusion of the Proof of Concept and that any test environments or capabilities will be deactivated after that. You should make clear to your Customer that they will be expected to provide feedback and confirmation around the scenario outcomes.

If it is clear that some criteria have not been met (or partially met), it is a good idea to dedicate some time in that meeting to review those areas in more detail: has the test been conducted correctly? Are these showstoppers or "nice to have" features?

If there is a gap, it could be that these features are in fact already in the roadmap for future delivery. If we can share this - you should verify internally with the Product team - then, by all means, acknowledge that the feature is not currently available, but is currently planned for delivery in the future. Of course, the level of commitment here will vary on a case by case basis.

After the meeting, update the document with the feedback and outcomes, including notes around who has provided that feedback during the meeting. This is because the document will become a deliverable that goes in hand with the Solution. Should the agreement conclude successfully, both the Customer and your organization will be able to use it as a baseline/blueprint for the final implementation. Share the final version of the document with the Customer, and once they are happy with it, change it to read-only mode.

Lastly, should there be disagreement raised at a later stage, we can discuss the matter with the Customer and reference back to our agreed documentation: it is not uncommon for the Customer implementation team (post-sales) to be different from the Customer evaluation team (pre-sales). Many times these discrepancies are mostly caused by poor communication or misunderstandings in handovers.

I don't know about you, but I think I have a really bad memory sometimes. This is why a document with all the key people, outcomes and milestones are noted down can be a great asset for both your organization and the Customer!

Summary

In this article, we have discussed what is a Proof of Concept, and why it is important to structure this high-effort activity so that it provides maximum value for both our Customer and our organization. I have shared what are, in my opinion, the key steps to a successful Proof of Concept, and some tips based on my experience.

If you haven't done so already, I recommend reading the previous article I wrote on the more general topic of qualification and scoping of opportunities: Tips on Technical Qualification and Scoping.

I hope you enjoyed reading it! If you have any comments, or if you would like to share your experiences at running PoCs, please feel free to leave a comment at the bottom of the page. I'll be glad to hear a few more "secret weapons" to run these more successfully. Thanks for your attention!

If you liked this article, follow me on Twitter for more updates!

Important: Please DONATE and help me raise funds to support the war refugees from 🇺🇦 Ukraine. Thank you! 🙏🏻

Comments

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Comments are moderated so there will be a delay before your comment is published.