Instant insights.

Petabyte scale.

In today’s data-centric world, it is crucial for businesses to be dynamic and adapt to ever-shifting competitions.

Ergo, gleaning insights at the speed
at which information is changing
is crucial.

Breakthroughs

Embracing the above premise, our powerful solution enables
businesses to address the challenge of navigating a nebulous market.

We do this by:  

01. Petabyte scale interactive analysis:
Addresses data-centric decision-making at scale. 

02. Genuine self-serve:
Enabling to glean insights. At the speed of change.  

03. Lowest TCO:
Open source platforms included.
Competitive within your budget.

Current obstacles to
timely data insights

Today big data analysis requires the help of an overworked data team with limited resources to process requests, resulting in an inordinate delay. Typically, most users fall into one or both of the following predicaments:

Predicament 1

Step 1. You send in a request for an analysis to the data team. 

Step 2. The data team has limited resources. 

Step. 3. Your request is dropped due to prioritization. 

Step. 4. You have no report and no insights. 

Predicament 2

Step. 1. The data team prioritizes your request in their queue. 

Step. 2. Despite prioritization, the delivery date is definitively open-ended.

Step 3. Delivery can be anywhere from a few-weeks to a few-months.

Step. 4. The delay translates to decision making sans insights. 

The vast majority of data requests fall within either of the above predicaments.
Decision making without data-driven insights is akin to shooting in the dark.

In light of this obsolescence of the current toolset to understand data at scale,
it is imperative that
a new environment has to be the bedrock for any big data analysis.

Decision Making

Using conversational search (e.g. Chat GPT, Gemini, CoPilot etc.)

A business person, visiting New Delhi, had to delay his stay by few days. Late that night, using conversational search, he was looking to see how to spend the time.

Suggestions were many, but after few refined subsequent questions, he narrowed his question to Mughal related sites.

Conversational search, besides giving him the options in New Delhi, also suggested that a trip to Agra to see Taj Mahal, which he had never imagined.

But, together with conversational search, the decision to see Taj Mahal was done.

This decision making was done by:

1.      Posing an intial question;

2.      Perusing the answer to see if the information can support making the decision;

3.      if not, modify or repose a question until enough information is found to make an informed decision or give up trying.

This process of data driven decision making is termed as “Intelligent decision making” where user and the computer work synergestically together to significantly enhance the decision making.

This was proposed many decades ago as “Intelligence Amplification” by D. Engelbart, W.Ashby, and others. <<< To go as a footnote

Observation on
Intelligent Decision Making

Three critical factors

User has to be able to:

1.      Pose questions by themselves (genuine self serve) over large corpus of data;

2.      Interactively review the answers to discern what is relevant and irrelevant;

3.      Repose questions interactively to follow the train of thought

If user relies on a data team to pose questions for conversation search,
and then wait many days to get answers back,
then intelligent decision making will fail due to the long delay.
Conversational search cannot be done over snail mail (or even emails).

We focused on the user-cloud interactions,
in describing the conversational search.
So, omitted the backend capabilities (e.g., LLM curation) that
enhances the intelligent decision making, even more.

But, we conclude that without the above factors,
intelligent decision making will fail, even with LLM advantages.

So these are critical factors for intelligent decision making.

Morphient’s
Intelligent Decision Making

This synergestic process between the user and the cloud can ONLY be successful due to the following technological capabilities of the environment:

  1. Interactively query and analyze unlimited data (e.g., PBs of data).
    So, users can follow their train of thoughts (speed-of-thought analysis).

  2. Custom visualization of the data to facilitate the recognition of the insight.

  3. Successively edit the query & analyze the data to confirm/reject the insight.

As mentioned above, these are critical capabilities for intelligent decision making.

Current analysis tools for big data are incapable of intelligent decision making even for TBs of data.

Capabilities needed for
Intelligent Decision Making

These following four capabilities are required for intelligent decision making:

  1. No Intermediary: Nobody (e.g. data team) is needed between the user and the data for user-interactions — including querying, analysis, modifying queries, optimizing queries, modifying visualization, etc.; even for business users.

  2. Genuine self-service: We use a document-model for all user-interactions to facilitate self-service. This mimics the WYSIWYG editors; i.e., sequence of edit operations for editing & creating a text document. Therefore, without any intermediary, user leverages the widely known mode of editing.

  3. Interactive analysis of unlimited data in a browser: Interactive analysis is assured by our query engine, in the browser, leveraging a custom optimized data strucutre (called JITQBE), mostly without going back to the cloud.

  4. Optimal parallel execution in the cloud: Occassionally a new JITQBE has to be computed in the cloud. By leveraging highly parallel executions (10’s of thousands of parallel executions) to quickly compute the JITQBE and return to the user, so the user can continue with their speed-of-thought analysis.

These are elaborated in subsequent sections.

Genuine Self Service

For empowering business users with intelligent decision making,
they need to query and analyze unlimited data by themselves.

This is done as follows:

  1. Visualization: In a browser, user views the data in a custom visualization — as a document.

  2. Modification: This document is edited akin to a text document (i.e., a sequence of edits with changes shown immediately) in order to modify the query or the visualization.

  3. View: This interactively-editable visualized-view of the data is the source of insight for the user.

  4. Model??: This view is referred to as story-as-a-document. Story refers to all possible analyses that can be done, using that view.

This story-as-a-document enables the business user to query & analyze by themselves without having to rely on the data team. Thus achieving genuine self service for the business user.

Data visualization products claims self-service feature that requires the data team to curate the subset of data which may take many days. This delay defeats the intent of self service. So, we refer to our offering as genuine self service to emphasize no-intermediary.<<<<< needs to put a footnote

Story-as-a-document

Story-as-a-document is viewed in a browser
for analysis and editing of the story as follows:

  1. Analyze: The user can interactively slice & dice the data rendered in the story.

  2. Interactivity: Interactive analysis is delivered by our query engine in the browser mostly without going back to the cloud.

  3. Modification: Editing of the story will change the story interactively by the query engine.

The story is used to browse/analyze the data as well as editing the story.
This assures that users can successively edit and peruse the story seamlessly.

Interactive Analysis
in a Browser

This story-as-a-document in a browser allows interactive analysis.
This tour de force is achieved as follows:

  1. Optimized Interactions: This interactive analysis is achieved by bringing the relevant curated subset of data in a data structure (called JITQBE), optimized for that specific story.

  2. JITQBE: JITQBE standing for Just-In-Time QBE (pronounced as cube) behaves like a data-cube without the exponential size problem.

  3. Engine: Query engine, in the browser, leverages the JITQBE to deliver interactive analysis experience.

  4. Recomputation of JITQBE: Editing of the story in the browser may automatically trigger recomputation of the JITQBE in the cloud. In matter of minutes, the user can interactively use the new edited story leveraging the new JITQBE.

The JITQBE, optimized for that story, assures interactive analysis for the user.
This level of interactivity assures that users can analyze at speed-of-thought.

Optimal Parallel Execution

Enabling analysis (i.e., JITQBE) over unlimited data as follows:

Computing JITQBE and any other processing over unlimited data requires parallel computing so that it will finish in time. For example when JITQBE is recomputed the user is waiting to continue the analysis. Finishing as quickly as possible is desirable.

  1. Goal driven Optimization: Leverage unlimited parallelism to achieve our goal.

  2. Cost effective computation: Using materialized JITQBE to compute other JITQBEs and using just-in-time provisioning of lambda to ramp up/down parallel computing allows us achieve the lowest TCO goal.

  3. Interactive experience: During ingestion of data materializing JITQBEs in anticipation of its usage allows delivering the interactive experience to the user.

All our computing in the cloud are achieved using unlimited parallel lambda execution. These lambdas are procured (and dismissed) just in time, which avoids the idling of expensive computing resources, to achieve our lowest TCO objective.

No Intermediary

Enabling genuine self service at lowest TCO as follows:

  1. Self service analysis: User does not depend on anybody to analyze the data.

  2. No tuning of data: Just-in-time recomputation of JITQBE eliminates the need for someone to optimize the execution over unlimited data.

  3. No maintenance: All cloud operations being run on lambda for a specific goal, which is monitored by Morphient. Therefore no one is needed for managing resources in the cloud.

The elimination of cutomer-intermediaries to operate the system allows Morphient SaaS to be used by business user with practically no IT involvement.

So, Instant Insight
@Unlimited Scale

Enabling analysis by business user at unlimited scale, as follows:

  1. Self-Service: Enabling users genuine self-service analysis at unlimited scale.

  2. Interactive experience: Enabling interactive analysis at unlimited scale.

  3. Server scaling: Enabling server computing at unlimited scale.

Morphient SaaS is capable of scaling Users, Stories, Data, Computing, etc..

Customer Success

Customer using it in production for four years, with following results:

  1. User Adoption: Over 150 users have opted to use it like smartphone app — NO manuals, NO training.

  2. Data Scaling: Over PB of data being analyzed.

  3. Server scaling: Many 10s of thousands of parallel lambdas executing with many 1000s of buckets of data for the required IOPs with no scaling limits in sight; thus establishing unlimited scalability.

Morphient SaaS is a proven product with years in production.

Free Trial Offer

There is no substitute to trying it on your own data.

  1. Email: Send us an email (jitqbe@morphient.com) to set up a video call.

  2. Data: Export your data in CSV or parquet format.

  3. Setup: Morphient will load your data and set up the SaaS in your own AWS. This typically takes 2 or 3 days.

  4. Try it: On board users and see for yourself.

We know you will be impressed.