Assessing Technology

Consider both functional requirements and non-functional requirements when you need to assess whether a technology will meet your team's needs or a business need.

Functional Requirements

The question you want to answer here is "can the technology perform its main job?"

Non-functional Requirements

The key question to answer here is "do I want to operate and maintain the technology?" I like to understand the things listed below when I check out a new technology.

Authentication & Authorization

• Determine how both humans and automated process will perform authentication and authorization.

• For humans, does the technology support your company's single-sign-on provider?

Availability

• Know whether the application is considered highly available or what you may need to do to achieve high availability.

Company Background

• Think twice before adopting a new, tiny company's application for an important need.

• How long has the company existed?

• Is the company profitable or healthy?

• How many employees does the company have?

• What does its roadmap look like?

Cost

• Someone has to pay the bills! What team's budget will be used to pay for this

• Understand how the cost works, such as per user, usage based, etc. Has that changed in the recent past?

Data In & Data Out Capabilities

• Consider both individual entities and mass data movement.

• Know what reporting capabilities come built-in to the application.

• Understand how to extract data for your data warehouse.

Deployment Pipelines

• Understand how to deploy multiple instances of the application in different environments (e.g., development, staging, and production).

• Know how you can define instances of the technology via version control, such as Terraform.

Development Language

• Know whether folks on your team have experience with the service's programming language.

Disaster Recovery and Backups

• Determine how you can recover from a catastrophic failure caused by a human.

Documentation

• The vendor should provide enough to get started and understand the application. And remember that no documentation is perfect.

Environments

• Know whether you can run multiple instances of the application or service in a development, staging, and production environment, for example.

Infrastructure Needs

• Understand what is needed for the application to run, such as in your own cloud provider, the company's cloud, etc.

Integration with other Services & Technologies

• See whether any integration is offered with other popular applications and services.

Latency / Physical Location

• Know whether you can run the technology near your other applications. Keep communication between applications is as fast as possible.

Legal Approval

• Ensure your company's legal team reviews the service's terms or in general, understand how your company onboards new software.

Monitoring & Alerting

• Understand how you will monitor the service and alert folks when it misbehaves.

Release Status

• Know the technology's maturity level such as alpha, beta, or generally available.

Reliability

• Ensure the application can handle hardware faults, how human errors can be undone, etc.

Scalability

• Understand how the technology scales both horizontally and vertically.

• Test how the technology behaves with more data, users, traffic, etc.

Security & Secrets Management

• Avoid storing any connection credentials and other sensitive information in version control. Ensure you can achieve authentication by in a safe manner without exposing credentials.

Support

• Review the company's support levels, their price, response time, and so on.

Upgrade Effort

• Understand how upgrades are performed, who performs them, how often they are required, whether downtime is needed, etc.

Usability and Limitations

• Know whether your resources are limited such as a max number of entities, API requests per second or day, etc.

Vendor

• If a vendor is implementing the technology for you, know whether they have done so before.