Big Data still in naivety proves Elusive to Big Picture and Randomness

Big Data as it exists today is not necessarily a tool that creates the Big Picture. Most systems that exists today are designed based on predicated methods. These rely on known variables, the only constraint unknown are the values of the variables. Predicated system are constrained to create newer insight as they are bounded by system boundaries defined by known variables. The new emergent variables can be considered as the non-causal agents until the newer order of the complex system begins to emerge; in which the newer variables manifests.

To develop wider and deeper insight about the behavior of the complex system as the system driven by entropy moves from one order to next; it does require to bring into its realm the possible newer variables. Lack of adequate techniques (Bayesian network included) in discerning all the variables that affect the system behavior creates uncertainty. Predicated system as such are blind sighted to such uncertainties. This creates inadequacies in methods when dealing with the randomness in the system. Emergence of newer variables that begin to affect the system behavior become the newer causes. Predicated system cannot hone on the variables that  lie dormant as the non-causal agents.

The hidden non-causal agents can be discovered by creating 2nd order generative semantic models. Predicative systems as such are not designed to dramatically alter its behavior; but only reflect the behavior constrained by those variables considered in its initial design

Big Data is conundrum unless Knowledge Management adds Semantics

Key to big data analytics, is to discover the underlying patterns in the businesses behaviors. When complexities and randomness increases, the interacting underlying patterns act to form generative orders, which become the emergent patterns.

Key to big data related architecture is to understand generative order and emergent architecture.

Generative Pattern

Generative Order, Randomness, Emergent

Complexity, Structure, Emergent Architecture
Complexity does not require a designer, rather randomness through recursive patterns generate complexities…the result is emergent. The system unassisted by outside sources, by increase in inherent randomness seeks order from generative process, hence it emerges. The architecture or structure of such system is termed “Emergent”


Christopher Alexander – Father of Pattern Language – Generative Pattern Key to Emergence


Discussion on Why Emphasis on ‘Problem Domain’ or ‘Requirement Management’ is very important in EA – Contextualization

Quality (Extract)

Alexander’s central premise, driving over thirty years of thoughts, actions, and writings, is that there is something fundamentally wrong with twentieth century architectural design methods and practices. In Notes, Alexander illustrates failures in the sensitivity of contemporary methods to the actual requirements and conditions surrounding their development. He argues that contemporary methods fail to generate products that satisfy the true requirements placed upon them by individuals and society, and fail to meet the real demands of real users, and ultimately fail in the basic requirement that design and engineering improve the human condition. Problems include:

  • Inability to balance individual, group, societal, and ecological needs.
  • Lack of purpose, order, and human scale.
  • Aesthetic and functional failure in adapting to local physical and social environments.
  • Development of materials and standardized components that are ill suited for use in any specific application.
  • Creation of artifacts that people do not like.

Timeless continues this theme, opening with phenomenologically toned essays on “the quality without a name”, the possession of which is the ultimate goal of any design product. It is impossible to briefly summarize this. Alexander presents a number of partial synonyms: freedom, life, wholeness, comfortability, and harmony. But no single term or example fully conveys meaning or captures the force of Alexander’s writings on the reader, especially surrounding the human impact of design, the feelings and aesthetics of designers and users, the need for commitment by developers to obtain and preserve wholeness, and its basis in the objective equilibrium of form. Alexander has been working for the past twelve years on a follow-up book, The Nature of Order, devoted solely to this topic (see [29,9]). 

On Generative Pattern – Discussion By Christopher Alexander


Patterns of Nature

Emergent Pattern

(How random events become recursive over time resulting into structured pattern formations. Another architectural paradox – ‘randomness’ and ‘pattern’.)