CafeIO

Deep Dive into Architecture Patterns, Part 2

Sun, 09 Apr 2023 09:00:00 +0530

Deep Dive into Architecture Patterns. This Post is focused on understanding different technology stacks and their usage.

This might very well be tangential to the idea of patterns, but I feel it is significant to be aware of the different technology stacks and from the core idea of patterns.

A technology stack refers to a set of technologies that are used together to create a software application or system. Different technology stacks are typically named after the key technologies that they include, such as the operating system, programming language, web server, and database.

Tech Stack Management

Some Popular Technology Stacks

Here’s a high-level view on some popular technology choices that we are currently presented with.

LAMP Stack : Linux: LAMP stack is based on the Linux operating system, which is an open-source operating system that is widely used for servers and desktops. Apache: Apache is a free, open-source web server software that is used to deliver web content over the internet. MySQL: MySQL is a popular open-source relational database management system (RDBMS) that is used to store and retrieve data. PHP: PHP is a popular open-source server-side scripting language that is used to create dynamic web pages and web applications.
MEAN Stack : MongoDB: MongoDB is a NoSQL document-oriented database that is used to store data in a flexible, JSON-like format. Express.js: Express.js is a lightweight web application framework that is used to build web applications and APIs in Node.js. AngularJS: AngularJS is a JavaScript-based front-end web application framework that is used to build dynamic and responsive user interfaces. Node.js: Node.js is a server-side JavaScript runtime environment that is used to run JavaScript on the server-side.
MERN Stack : MongoDB: Same as MEAN stack, MongoDB is used as the database in MERN stack.Express.js: Same as MEAN stack, Express.js is used as the server-side framework.React.js: React.js is a popular front-end JavaScript library that is used to build user interfaces.Node.js: Same as MEAN stack, Node.js is used as the server-side runtime environment.
MERN with Next.js Stack : MongoDB: Same as MERN stack, MongoDB is used as the database in this stack.Express.js: Same as MERN stack, Express.js is used as the server-side framework.React.js: Same as MERN stack, React.js is used as the front-end JavaScript library. Node.js: Same as the MERN stack, Node.js is used as the server-side runtime environment.Next.js: Next.js is a popular React-based framework that is used to build server-side rendered web applications.
.NET Stack : .NET Framework or .NET Core: .NET Framework is a free, open-source development platform for building Windows-based desktop and web applications. .NET Core is a newer version of the framework that is designed to be cross-platform.C#: C# is a modern, object-oriented programming language that is used to develop applications on the .NET platform. Visual Studio IDE: Visual Studio is an integrated development environment (IDE) that is used to develop, test, and deploy .NET applications.SQL Server: SQL Server is a popular relational database management system (RDBMS) developed by Microsoft.
Python Flask Stack : Python: Python is a high-level programming language that is used for web development, data analysis, and artificial intelligence. Flask: Flask is a lightweight web application framework that is used to build web applications and APIs in Python. PostgreSQL: PostgreSQL is a popular open-source relational database management system (RDBMS) that is used to store and retrieve data.
Django Stack : Django: Django is a high-level Python web framework that is used to build web applications and APIs quickly and efficiently. PostgreSQL: PostgreSQL is a popular open-source relational database management system (RDBMS) that is used to store and retrieve data. Nginx: Nginx is a high-performance web server that is used to deliver web content over the internet.
Java Stack : Java: Java is a widely used programming language that is used to develop a wide range of applications, from desktop and mobile apps to web and enterprise applications.Spring Framework: Spring is a popular open-source framework that is used to build web applications and enterprise-level software systems. Hibernate: Hibernate is an object-relational mapping (ORM) framework that is used to manage database operations in Java-based applications. Apache Tomcat: Apache Tomcat is a popular open-source web server and servlet container that is used to deploy and run Java-based web applications.
Serverless Stack : AWS Lambda or Azure Functions: Serverless stack is a technology stack that doesn’t require any server management or infrastructure. AWS Lambda and Azure Functions are popular serverless computing services offered by Amazon Web Services and Microsoft Azure, respectively._ Amazon S3 or Azure Blob Storage_: Amazon S3 and Azure Blob Storage are popular object storage services that can be used to store and retrieve data in a serverless environment._ Amazon API Gateway or Azure API Management:_ Amazon API Gateway and Azure API Management are popular API gateway services that can be used to manage and secure APIs in a serverless environment.

Factors to keep in mind before selecting Tech Stack

When selecting a technology stack, there are several factors that need to be considered. Here are some key factors that should be kept in mind:

Project requirements: Consider the requirements of the project, including the type of application, the features it needs to support, and the expected usage. The technology stack should be able to support these requirements and provide the necessary functionality.
Scalability: Consider the scalability requirements of the project, including the expected growth of the application over time. The technology stack should be able to scale to meet the demands of the application without sacrificing performance.
Cost: Consider the budget for the project and the cost of the technology stack. Some technology stacks may require more expensive hardware or licensing fees, while others may be more cost-effective.
Development resources: Consider the skillset and experience of the development team. The technology stack should align with the team’s expertise to ensure efficient development and maintenance.
Community support: Consider the level of community support for the technology stack. A strong community can provide valuable resources, tools, and support for developers, as well as ensure the longevity and sustainability of the technology.
Integration: Consider the integration requirements of the project. The technology stack should be able to integrate with other systems and tools as needed.
Security: Consider the security requirements of the project, including data privacy, regulatory compliance, and potential vulnerabilities. The technology stack should provide robust security features and be regularly updated to address new security threats.

Overall, it is important to carefully evaluate the specific requirements and constraints of the project to select the most appropriate technology stack.

Some popular technology stacks mentioned are LAMP, MEAN, MERN, MERN with Next.js, .NET, Python Flask, Django, Java, and Serverless stack. Factors to consider when choosing a technology stack include project requirements, scalability, cost, and development resources.

Deployment Patterns

Deployment Approaches

The deployment options available for a technology stack depend on the specific components of the stack and the requirements of the project. However, here are some common deployment options that are available for most technology stacks:

On-premises deployment: This is a traditional deployment method where the application is deployed on hardware that is located on-site or within the organization’s premises. This deployment method requires the organization to manage the hardware, software, and security of the application.
Virtual machines (VM) deployment: In this deployment method, the application is deployed on a virtual machine that is hosted on a public cloud or private infrastructure. This method provides greater flexibility than on-premises deployment and allows the organization to scale resources up or down as needed.
Container deployment: Container deployment involves deploying the application in a container, such as Docker, which encapsulates the application and its dependencies. This method provides greater portability, as the container can be deployed on any infrastructure that supports the container runtime.
Serverless deployment: In a serverless deployment, the application is deployed as a set of functions that run in a serverless computing environment, such as AWS Lambda or Azure Functions. This deployment method eliminates the need to manage the underlying infrastructure and provides greater scalability.
Platform as a Service (PaaS) deployment: PaaS deployment involves deploying the application on a cloud-based platform, such as Google App Engine, Heroku, or Microsoft Azure App Service. The platform manages the underlying infrastructure and provides developers with a pre-configured environment for deploying and running their applications.
Infrastructure as a Service (IaaS) deployment: IaaS deployment involves deploying the application on a cloud-based infrastructure, such as Amazon Web Services (AWS) or Microsoft Azure. This deployment method provides organizations with greater control over the underlying infrastructure and allows them to customize the environment to meet their specific requirements.

Factors Influencing Deployment Strategies

When selecting a deployment method for a technology stack, there are several factors that should be considered. Here are some key choices that should be made:

Scalability: Consider the expected traffic and usage of the application. If the application is expected to experience high levels of traffic or usage, a scalable deployment method such as serverless or container deployment may be more appropriate.
Cost: Consider the budget for the project and the cost of the deployment method. For example, on-premises deployment may be more cost-effective for small projects, while cloud-based deployment may be more cost-effective for larger projects.
Flexibility: Consider the level of flexibility required for the project. For example, if the application needs to be deployed in different environments, a container or serverless deployment may be more appropriate.
Maintenance: Consider the level of maintenance required for the deployment method. For example, on-premises deployment may require more maintenance than cloud-based deployment, which is managed by the cloud provider.
Security: Consider the security requirements for the project. For example, if the application handles sensitive data, a cloud-based deployment method may require additional security measures to ensure the data is protected.
Integration: Consider the integration requirements for the project. For example, if the application needs to integrate with other systems, a platform as a service (PaaS) deployment method may be more appropriate, as it provides pre-configured environments for deploying and running applications.

Overall, it is important to carefully evaluate the specific requirements and constraints of the project to select the most appropriate deployment method for the technology stack.

The section discusses different deployment approaches available for technology stacks, including on-premises, virtual machines, container, serverless, PaaS, and IaaS deployment. It also highlights the factors that should be considered when selecting a deployment method, such as scalability, cost, flexibility, maintenance, security, and integration requirements. The selection of the deployment method should depend on the specific requirements and constraints of the project.

Summary

In summary, we discussed the popular technology stacks such as LAMP, MEAN, MERN, MERN with Next.js, .NET, Python Flask, Django, Java, and Serverless stack. When choosing a technology stack, it is important to consider project requirements, scalability, cost, and development resources.

Deployment approaches are also important when it comes to technology stacks. There are several deployment methods available, including on-premises, virtual machines, container, serverless, PaaS, and IaaS deployment. Each deployment method has its own advantages and disadvantages. For example, serverless deployment eliminates the need to manage underlying infrastructure and provides greater scalability. The selection of the deployment method should depend on the specific requirements and constraints of the project. When selecting a deployment method, scalability, cost, flexibility, maintenance, security, and integration requirements should be considered.

In summary, the selection of the technology stack and deployment method should depend on the specific requirements and constraints of the project. It is important to carefully evaluate the project’s needs and select the appropriate technology stack and deployment method. By doing so, you can ensure that the project is successful and meets its goals

Unlocking The Secrets of Algo Trading: Learn How to Use Python for Getting Data and Strategies…!!

Tue, 04 Apr 2023 09:00:00 +0530

With Python’s limitless libraries and tools, we can build a robust foundation to predict market trends, analyse financial data, and create advanced financial models.

Hey there! In this post, I’m going to talk about setting things up for algo trading. In my previous video, I covered many aspects of algo trading, including technology, tools, markets, and data. I will build on top of that in this blog post. I would like to point out that this post is accompanied by a video which I shared before.

You can watch the supplementary video and Follow Along the Video.

Hardware & Software Setup

You don’t need a fancy computer for this, anything online or a simple computer will work.

8 GB of RAM with an i5 Processor. Something lower might work too.
Apple Silicon (m1/m2) is great too if you can install the required dependencies. There are few libraries which do not work on it.
Anaconda Environment with Python 3.7. Though higher version of python can be taken, however I have found this to be the common denominator which most libraries support.
List of Libraries to Install
- Pandas
- Numpy
- Matplotlib, Seaborn, Plotly
- yfinance
- Cufflinks
Bonus : Just use Google Colab and install yfinance, Cufflinks in there and get started

Here’s what we will focus on

We will be setting up a python environment, getting market data using Yahoo Finance, and doing some basic statistical analysis. We will then create a simple strategy and compare returns.

Get market data from Yahoo Finance.
Play with the data, get data from NSE, and do stats on it.
Calculate moving averages and do plotting.
Try to do something known as Bollinger bands, which is a technical indicator.
Make a simple moving average-based strategy.

More Details

After installing the libraries, we will use Yahoo Finance to get market data. We will define a variable ticker and set start and end dates. Then, we will download the data and create a data frame. We will also reduce the number of days to make it easier to plot.

Next, we will plot the high, lows, and search for patterns. We will also calculate some basic formulas like simple moving average, exponential moving averages, and Bollinger bands. Bollinger bands is a plus-minus two standard deviations up and down from a simple moving average.

Finally, we will make a simple moving average-based strategy. The strategy will be to sell if a condition is met and hold if it is not.

The Complete Code is shared below

Install necessary libraries.
Define variable ticker and set start and end dates.
Download data and create a data frame.
Reduce the number of days to make it easier to plot.
Plot high lows and search for patterns.
Calculate basic formulas like simple moving average, exponential moving averages, and Bollinger bands.

Working with a Strategy

In the world of algorithmic trading, having a well-defined strategy is crucial for success. A trading strategy is simply a set of rules that traders follow to generate signals that indicate when to buy, sell, or do nothing.

One popular trading strategy is using the simple moving average (SMA). The SMA is a technical indicator that helps traders identify trends in the market. It calculates the average price of a security over a specific time period, such as 12 days, by adding up the closing prices of each day and dividing it by the number of days in the period.

When using the SMA, traders typically buy a stock when its closing price is higher than its simple moving average and sell it when the closing price is lower. This approach can help traders identify potential uptrends or downtrends in the market and make informed decisions on when to enter or exit a trade.

However, it’s important to note that no trading strategy is foolproof. Market conditions can change rapidly, and unforeseen events can impact a stock’s performance. Therefore, it’s essential to continually monitor the market and adjust trading strategies as needed.

In addition to the SMA, there are many other technical indicators and trading strategies that traders can use to make informed decisions in the market. Some popular ones include the relative strength index (RSI), the moving average convergence divergence (MACD), and the Bollinger Bands.

When studying financial engineering and data science, it’s essential to clean up data to ensure that the calculations are accurate. This often involves dropping unnecessary columns and removing any errors or outliers that may skew the results.

In summary, having a well-defined trading strategy is essential for success in algorithmic trading. The SMA is just one of many technical indicators and trading strategies that traders can use to make informed decisions in the market. However, it’s important to continually monitor the market and adjust strategies as needed to stay ahead of the game.

Complete Code

#pip install yfinance plotly cufflinks

# Core Imports
import pandas as pd
import numpy as np
import datetime
import yfinance as yf
import matplotlib.pyplot as plt

#Ticker is the stock that we want to get
Ticker = "^NSEI"


# Get Sample data based on a Ticker
end1 = datetime.date.today()
start1 = end1 - pd.Timedelta(days=5)

df = yf.download(Ticker, start=start1, end=end1, interval="5m" )
print(df.head())
df.info()

# Plotting the Data

df1a = df.copy()
df1a.loc['2023-02-13', ['Open', 'High', 'Low', 'Close']].plot(grid=True, linewidth=1, figsize=(14, 9))

# Just the Closing Price
df1a['Close'].plot(grid=True, linewidth=1, figsize=(14, 9))

# Core Calculation Functions 
# SMA
def get_sma(prices, rate):
  return prices.rolling(rate).mean()

# EMA
def get_ema(prices, rate):
  return prices.ewm(span=ema, adjust=False).mean()

# Bollinger Bands

# Bollinger bands are a technical analysis tool used by traders to identify potential entry and exit points in the market. They are created by plotting a moving average of the price along with two standard deviation lines above and below it. By doing so, Bollinger bands can provide an indication of whether a stock is overbought or oversold. They also help traders identify potential breakouts or reversals in the market. With this knowledge, traders can make more informed decisions when entering or exiting a position in the market.

def get_bollinger_bands(prices,rate):
    sma = get_sma(prices, rate)
    std = prices.rolling(rate).std()
    bollinger_up = sma + std * 2 # Calculate top band
    bollinger_down = sma - std * 2 # Calculate bottom band
    return bollinger_up, bollinger_down


# Other Functions

def download_daily_data(ticker, start, end):
    """ 
    The function downloads daily market data to a pandas DataFrame 
    using the 'yfinance' API between the dates specified.
    """
    data = yf.download(ticker, start, end)
    
    return data

def compute_daily_returns(data):
    """ 
    The function computes daily log returns based on the Close prices in the pandas DataFrame
    and stores it in a column  called 'cc_returns'.
    """
    data['cc_returns'] = np.log(data['Close'] / data['Close'].shift(1))
    return data

# Generate Bollinger bands for the above

bollinger_up, bollinger_down = get_bollinger_bands(df1a['Close'],20)


# Plot the Results

symbol = 'NSE'
closing_prices = df1a['Close']

plt.title(symbol + ' Bollinger Bands')
plt.xlabel('Days')
plt.ylabel('Closing Prices')
plt.plot(closing_prices, label='Closing Prices')
plt.plot(bollinger_up, label='Bollinger Up', c='g')
plt.plot(bollinger_down, label='Bollinger Down', c='r')
plt.legend()
plt.show()


# Strategy Perform Calculations

# Remove the Un-necessary Columns
df1a.drop(columns=['High', 'Low', 'Volume'], inplace=True)

# Create a new colum which captures the percentage cahnge from previous day
df1a['cc_returns'] = df1a['Close'].pct_change()

# Define a short 12 day sma 
sma = 12
df1a['sma'] = df1a['Close'].rolling(window=sma).mean()

print(df1a.head())
print(df1a.tail())

# If the closging price is higher that sma buy else do nothing
df1a['position'] = np.where((df1a['Close'] > df1a['sma']), 1, 0)
df1a['position'] = df1a['position'].shift(1)
df1a['position'].value_counts()

# Plotting the above strategy returns

df1a['strategy_returns'] = df1a['cc_returns'] * df1a['position']


df1a['strategy_returns'] = 1 + df1a['strategy_returns']
df1a['cc_returns'] = 1 + df1a['cc_returns']

print(df1a.head())
print(df1a.tail())

df1a[['cc_returns', 'strategy_returns']].cumprod().plot(grid=True, figsize=(9, 5))

print('Buy and hold returns: ', np.round(df1a['cc_returns'].cumprod()[-1], 2))
print('Strategy returns: ', np.round(df1a['strategy_returns'].cumprod()[-1], 2))
HyperWrite Logo

Deep Dive into Architecture Patterns, Part 1

Sun, 02 Apr 2023 09:00:00 +0530

Deep Dive into Architecture Patterns. This Post is focused on understanding requirements (Functional and Non-Functional)

Architecture patterns are general, reusable solutions to recurring design problems at the system level. They provide a blueprint for organising and structuring software systems and their components, such as modules, layers, and services. Architecture patterns provide guidance on how to design and implement software systems with specific characteristics and qualities, such as scalability, performance, security, and maintainability.

In the previous post Architecture Patterns, we looked at patterns from a bird’s-eye view. Patterns, however, are the heart and soul of Architecture and warrant a much deeper conversation. In the upcoming parts (still discovering how many to write) I intend to talk about patterns in more depth.

In this post, I will talk about System requirement that lead to the right pattern selection.

Factors Influencing Architectural Pattern Choice

There could be a variety of factors influencing the choice of patterns. Some of the more obvious ones are listed below.

Functional Requirements : Functional requirements describe what the system should do, such as the features it must provide and the tasks it must perform. The functional requirements of a system can help determine the appropriate architecture pattern. For example, if the system needs to support multiple user interfaces, a layered architecture pattern may be appropriate, as it provides a clear separation between the presentation layer and the business logic layer. On the other hand, if the system needs to support complex business workflows and data processing, a workflow-driven architecture pattern may be more suitable.
Technology Stack: The technology stack used in the system can influence the choice of architecture pattern. For example, if the system is built using a specific programming language or framework, there may be architecture patterns that are better suited to that technology stack.
Domain Complexity: The complexity of the domain that the system is intended to model can help determine the appropriate architecture pattern. For example, if the domain is complex and involves many interacting components, a domain-driven design (DDD) architecture pattern may be appropriate.
Non-functional Requirements: Non-functional requirements such as performance, security, and maintainability can also influence the choice of architecture pattern. For example, if the system needs to be highly secure, a layered architecture pattern may be appropriate.

Beyond this, I have found the following to be also a contributing factor :

Team’s Capability (One of the Underdog in the selection)
COTS products and Vendor-driven environments (Influence Security and compliance heavily)
Existing IT Landscape (Data Center Setups can potentially challenge some modern patterns which are better suited to cloud)
Pricing and Time To Market

Functional Requirements

Functional requirements are a crucial element in the development of software systems. They define what a system should do, including its features and functionality. When it comes to designing the software architecture, functional requirements have a significant impact on the selection of appropriate architecture patterns.

Functional requirements can determine the appropriate architecture pattern by providing insights into the system’s features and capabilities.

For example, if the system needs to handle complex business workflows and data processing, a workflow-driven architecture pattern may be more suitable. This pattern provides a visual representation of the workflow and ensures that the business logic is decoupled from the presentation layer.
Similarly, if the system needs to support multiple user interfaces, a layered architecture pattern may be appropriate. This pattern separates the system into different layers, each with a specific responsibility, making it easier to maintain and scale the system.
Another example of how functional requirements can impact architecture patterns is in the case of real-time systems. Real-time systems have stringent timing requirements and require an architecture pattern that can provide predictable and reliable performance. In this case, a microservices architecture pattern may be more suitable, as it enables the system to be broken down into smaller, independent services that can be scaled and deployed separately.

In conclusion, functional requirements have a significant impact on the selection of appropriate architecture patterns. The selection of an architecture pattern must meet the functional requirements of the system and ensure that it is scalable, maintainable, and extensible. Therefore, it is crucial to carefully analyse the system’s functional requirements before selecting an appropriate architecture pattern.

Non – Functional requirements

Non-functional requirements define the performance, security, reliability, and other quality attributes that a software system must meet. Non-functional requirements are important because they determine how well the system performs its tasks and how easy it is to use and maintain.

One example of how non-functional requirements impact architecture patterns is in the case of performance. If a system has high-performance requirements, such as low latency or high throughput, then an appropriate architecture pattern must be selected. In this case, a distributed architecture pattern, such as a microservices architecture or a service-oriented architecture, may be more suitable, as it provides scalability and fault tolerance.
Similarly, if a system needs to be highly secure, a layered architecture pattern with a clear separation of concerns may be more appropriate. This pattern separates the system into different layers, each with a specific responsibility, making it easier to secure and maintain.
Another example of how non-functional requirements can influence architecture patterns is in the case of scalability. If a system needs to handle a large volume of data or traffic, then an appropriate architecture pattern must be selected. In this case, a distributed architecture pattern such as a microservices' architecture may be more suitable, as it enables the system to be broken down into smaller, independent services that can be scaled and deployed separately.

Types of NFRs

In general, NFRs are quality attributes of the system and are usually the “ity” requirements. Some of them are mentioned before.

Performance: This refers to how the system responds to a specific workload or user load. Performance requirements can include response time, throughput, and resource utilization.
Scalability: This refers to the system’s ability to handle an increasing amount of work or users without affecting its performance. Scalability requirements can include horizontal scalability, vertical scalability, and load balancing.
Reliability: This refers to the system’s ability to function correctly and without failure over time. Reliability requirements can include fault tolerance, disaster recovery, and backup and recovery.
Security: This refers to the system’s ability to protect against unauthorised access, data theft, and other security threats. Security requirements can include authentication, access control, encryption, and compliance with industry standards.
Usability: This refers to how easy the system is to use and how well it meets user needs. Usability requirements can include accessibility, responsiveness, and user interface design.
Maintainability: This refers to how easy it is to maintain and update the system over time. Maintainability requirements can include modularity, documentation, and code quality.
Interoperability: This refers to the system’s ability to interact with other systems and applications. Interoperability requirements can include support for industry standards and protocols, data exchange formats, and integration with third-party systems.
Compatibility: This refers to the system’s ability to work with different hardware, software, and network environments. Compatibility requirements can include support for different operating systems, browsers, databases, and other software components.
Availability: This refers to the system’s ability to be accessible and operational when required. Availability requirements can include uptime, downtime, and recovery time objectives.
Capacity: This refers to the system’s ability to handle a specific volume of data or transactions. Capacity requirements can include data storage, processing power, and network bandwidth.
Compliance: This refers to the system’s ability to meet legal, regulatory, and industry standards. Compliance requirements can include data privacy, security regulations, accessibility standards, and industry-specific regulations.
Performance efficiency: This refers to the system’s ability to achieve performance objectives with minimal resource consumption. Performance efficiency requirements can include energy efficiency, resource utilization, and optimization of memory and processing power.
Testability: This refers to how easy it is to test the system and validate its functionality. Testability requirements can include support for automated testing, test data management, and traceability of test results.

The list can be further extended, but this is a good starting point for evaluation of NFRs.

How to Prioritise NFRs?

Prioritising non-functional requirements (NFRs) is critical to the success of software development projects. Here are some common prioritisation techniques for NFRs:

MoSCoW method: This technique involves dividing NFRs into four categories: Must have, Should have, Could have, and Won’t have. The must-have requirements are the most critical, while the won’t-have requirements are the least important.
Cost-benefit analysis: This technique involves assessing the cost of implementing an NFR against its benefits. NFRs with the highest benefits and lowest costs are given a higher priority.
Risk-based prioritisation: This technique involves identifying the potential risks associated with each NFR and prioritizing them based on their impact on the system’s performance, reliability, and security.
User-based prioritisation: This technique involves prioritizing NFRs based on user needs and preferences. User feedback and surveys can help identify the most critical NFRs from a user’s perspective.
Impact mapping: This technique involves mapping NFRs to the system’s goals and objectives. NFRs that have a more significant impact on achieving the system’s goals are given a higher priority.
Kano model: This technique involves categorising NFRs into three categories: must-haves, performance, and delighters. Must-haves are essential requirements, performance requirements are expected by users, and delighters are features that exceed user expectations.
Planning poker: This technique involves a collaborative effort where development team members discuss and estimate the priority of each NFR. This technique can help achieve consensus and ensure that.

In conclusion, prioritisation of NFRs is a critical activity in software development. By using techniques such as MoSCoW, AHP, Cost-Benefit Analysis, Risk-Based Prioritisation, Agile Prioritisation, and Kano Model, software developers can ensure that the most critical requirements are met, and resources are allocated effectively to achieve the desired system performance, reliability, and user satisfaction.

Summary

“Deep Dive into Architecture Patterns, Part 1” is a comprehensive guide that delves into the world of architecture patterns. Architecture patterns provide reusable solutions to common design problems, serving as a blueprint for structuring and organising software systems and their components. The post emphasises understanding the functional and non-functional requirements when selecting the appropriate architecture pattern for a system.

Functional requirements define what the system should do, such as the features it must provide and the tasks it must perform. These requirements can help determine the appropriate architecture pattern. For example, a workflow-driven architecture pattern is more suitable for systems that handle complex business workflows and data processing. On the other hand, a layered architecture pattern may be more appropriate for systems that support multiple user interfaces.

Non-functional requirements, such as performance, security, reliability, and scalability, determine how well the system performs its tasks and how easy it is to use and maintain. A distributed architecture pattern, such as a microservices' architecture, may be more suitable for systems that require high-performance and scalability. A layered architecture pattern with a clear separation of concerns may be more appropriate for highly secure systems. The post also discusses the types of NFRs, such as performance, security, and maintainability.

The post concludes that selecting the appropriate architecture pattern requires careful analysis of the system’s functional and non-functional requirements to ensure that it is scalable, maintainable, and extensible. Other factors such as technology stack, domain complexity, and team capability can also influence the pattern selection. This informative guide is a must-read for developers, architects, and software engineers who are keen to learn more about architecture patterns and their impact on software design.

References

https://akfpartners.com/growth-blog/the-problem-with-non-functional-requirements

Why is it Important to understand TimeSeries in Algo Trading?

Tue, 28 Mar 2023 09:00:00 +0530

Time series analysis plays an important role in understanding and forecasting stock market behaviour

What is Algorithmic Trading?

Algorithmic trading, also known as algo trading, is a method of executing trades in financial markets using pre-programmed instructions that automatically execute trades based on certain criteria or market conditions.

Algo trading is commonly used by institutional investors such as hedge funds, banks, and pension funds, as well as by individual traders. The use of algorithms can lead to increased efficiency and speed of trading, as well as reduced transaction costs. However, algo trading can also come with risks, such as the potential for errors in the programming or unexpected market movements that could trigger unintended trades.

What is Time Series?

In the context of the stock market, a time series is a set of historical stock market data that is collected and organised over a period of time. This data typically includes the prices of a particular stock, index, or other financial instrument at different points in time, and can also include other metrics such as trading volume or market capitalisation.

If you’re interested in trading, then you’ve likely heard of algorithmic trading, or algo trading for short. This approach uses sophisticated mathematical models and algorithms to analyze market data and identify buying and selling opportunities automatically. By doing so, algo trading eliminates the need for human intervention in executing trades.

When it comes to analyzing stock market data, one popular method is time series analysis. This technique involves studying historical data to identify patterns, trends, and relationships between different variables.

Technical analysts often use time series data to develop trading strategies and make predictions about future market movements. By using tools such as moving averages, chart patterns, and technical indicators, traders can identify patterns and trends in the data and make informed decisions about buying, selling, or holding stocks or other financial instruments.

By leveraging the power of algo trading and time series analysis, traders can gain a more profound understanding of the stock market and make more informed investment decisions.

Components of Stock Market Data

Stock market data typically includes a variety of components that can provide insights into the performance of individual stocks, as well as broader market trends. Some of the most common components of stock market data include:

Stock prices: The prices of individual stocks at different points in time are a key component of stock market data. This includes the opening and closing prices, as well as intra-day highs and lows.
Volume: The total number of shares traded for a particular stock over a given time period is known as volume. Volume can be an important indicator of market sentiment and can help investors understand the level of interest in a particular stock.
Market capitalisation: Market capitalisation is the total value of a company’s outstanding shares. It is calculated by multiplying the total number of shares by the current market price per share.
Dividends: Dividends are payments made by companies to their shareholders as a share of profits. Dividend data can provide insights into a company’s financial health and growth potential.
Earnings: Earnings reports provide information on a company’s financial performance, including revenues, expenses, and net income. This information can be used to evaluate the financial health and growth potential of a company.
News and events: News and events that impact the stock market, such as company announcements, economic reports, and geopolitical events, can also be an important component of stock market data.

By analyzing these components and searching for patterns and trends, investors and traders can gain a more profound understanding of the stock market and make more informed investment decisions.

If you play close attention, most of these components are time—varying. Typically, Period and Interval are two aspects of time which are factored in. For example,

Stock Closing Price for a year with the interval being 1 day
Daily Volume for # of traded days
Ticker data (almost real-time or Second internal
Annual Earning Reports

Components of Time Series

By identifying these components in a time series, analysts can gain a more in-depth understanding of the underlying patterns and trends in the data.

This information can be used to make predictions, develop forecasting models, and inform decision-making for Quantitative finance.

Trend: A trend is a long-term increase or decrease in the data over time. It can be upward, downward, or stable.
Seasonality: Seasonality refers to the regular and periodic fluctuations in the data that occur within a year or other fixed time period. For example, sales of winter clothing may be higher during the winter months, while sales of summer clothing may be higher in the summer months.
Cyclical components: Cyclical components are fluctuations in the data that occur over a period longer than a year. They can be influenced by factors such as economic cycles or business cycles.
Irregular components: Irregular components are unexpected fluctuations in the data that are not accounted for by trend, seasonality, or cyclical components. They may be due to random events, measurement error, or other factors.

Putting Theory to Practise

We will look at two simple strategies that are purely time series based and are extremely popular in the trading world. I would like to point out that this is not a trading advice and the examples just illustrate the concept.

Golden Crossover strategy

The Golden Crossover Strategy is a popular technical analysis trading strategy used in the stock market. It involves using two moving averages of different lengths to identify buying and selling signals.

The two moving averages used in the Golden Crossover Strategy are the 50-day moving average and the 200-day moving average.
The 50-day moving average is calculated by taking the average price of a stock over the last 50 days, while the 200-day moving average is calculated by taking the average price of a stock over the last 200 days.

The Golden Crossover occurs when the 50-day moving average crosses above the 200-day moving average, indicating a bullish signal. This is interpreted as a buy signal, as it suggests that the stock is trending upwards and has the potential to increase in value.

import yfinance as yf
import numpy as np

# Download historical stock data for a given ticker symbol
stock_data = yf.download('AAPL', start='2020-01-01', end='2022-01-01')

# Calculate the 50-day and 200-day moving averages
stock_data['50_day_ma'] = stock_data['Adj Close'].rolling(window=50).mean()
stock_data['200_day_ma'] = stock_data['Adj Close'].rolling(window=200).mean()

# Generate signals based on the Golden Crossover strategy
stock_data['signal'] = 0.0
stock_data['signal'][50:] = np.where(stock_data['50_day_ma'][50:] 
                                      > stock_data['200_day_ma'][50:], 1.0, 0.0)

# Calculate the positions based on the signals
stock_data['position'] = stock_data['signal'].diff()

# Plot the stock prices, moving averages, and trading signals
import matplotlib.pyplot as plt

plt.figure(figsize=(10,5))
plt.plot(stock_data['Adj Close'], label='AAPL')
plt.plot(stock_data['50_day_ma'], label='50-day MA')
plt.plot(stock_data['200_day_ma'], label='200-day MA')

# Plot the buy and sell signals
plt.plot(stock_data[stock_data['position'] == 1].index, 
         stock_data['50_day_ma'][stock_data['position'] == 1], 
         '^', markersize=10, color='green', label='buy')
plt.plot(stock_data[stock_data['position'] == -1].index, 
         stock_data['50_day_ma'][stock_data['position'] == -1], 
         'v', markersize=10, color='red', label='sell')
plt.xlabel('Date')
plt.ylabel('Price')
plt.title('Golden Crossover Strategy for AAPL')
plt.legend()
plt.show()

ARIMA Model (Autoregressive Integrated Moving Average)

The ARIMA model consists of three components: autoregression, integration, and moving average. Autoregression refers to the use of past values of the time series as predictors for future values. Integration involves transforming the data to make it stationary, meaning that the mean and variance are constant over time. Moving average involves using past errors as predictors for future values.

To use ARIMA for algo trading, historical stock price data is first analyzed to identify the trend, seasonality, and other components of the time series. The data is then transformed to make it stationary, and an ARIMA model is fit to the transformed data. The model can be used to forecast future stock prices, and these forecasts can be used to make trading decisions automatically.

Summary

Algorithmic trading is the use of pre-programmed instructions to automatically execute trades in financial markets based on certain criteria or market conditions. It is popular among institutional investors and individual traders for its speed and efficiency, although it also comes with risks. Time series, on the other hand, refers to a set of historical stock market data collected and organised over a period of time, typically including stock prices and trading volumes. Technical analysts use time series analysis to identify patterns and trends and develop trading strategies.

Components of stock market data include stock prices, volume, market capitalisation, dividends, earnings, and news and events, all of which can provide insights into the performance of individual stocks and broader market trends. In analyzing these components and searching for patterns and trends, investors and traders can gain a more profound understanding of the stock market and make more informed investment decisions. Time series analysis helps analysts identify the components of a time series, such as trend, seasonality, cyclical components, and irregular components, which can inform decision-making in quantitative finance.

The article also includes examples of two simple trading strategies based on time series analysis: the Golden Crossover Strategy and the Momentum Strategy. These strategies are not intended as trading advice but are used to illustrate the concept of time series analysis in trading.

References

Architecture Patterns?

Sun, 26 Mar 2023 09:00:00 +0530

Do software products follow patterns? Why is it important to think about them?

Introduction

Human beings have a knack for finding patterns. Given clouds we tend to find shapes and faces in them, we connect the stars with imaginary lines to define autonomic shapes etc. Seeing patterns is a natural function of the human brain intended to help us learn.

Software products are no different.

**Design problems **arise during the software development process when there is a need to make decisions about the software architecture, design, and implementation.

Examples of design problems may include

Choosing the appropriate software components and technologies,
Selecting the most suitable algorithms and data structures,
Optimising performance
Ensuring maintainability and scalability
Balancing competing requirements.

Unfortunately, these discussions are extremely common and difficult to address given that software systems are in a constant state of evolution. The subsequent decisions have to be made by balancing trade-offs between different factors, such as performance, scalability, security, maintainability, and usability.

Patterns to the Rescue

Fortunately, for us, these design concerns have become repetitive over time and there are well established constructs that either solve them or provide a conversation ground for. These are called Architecture Patterns.

In a nutshell, An architecture patters is a general, reusable solution to a recurring design concern in software architecture. It provides a standard template for solving a particular architectural concern, and it can be applied across different systems and contexts.

Architecture patterns are usually higher-level than Design patterns (Future Post, most likely), which focus on individual software components and their interactions. They help to ensure that software systems are designed with consistency, maintainability, and scalability in mind.

By using architecture patterns, software architects can avoid reinventing the wheel for common design challenges and instead rely on proven solutions.

Some examples of architecture patterns include:

Layered architecture
Client-server architecture
Microservices architecture
Model-View-Controller (MVC) architecture
Event-driven architecture

Layered Architecture

also known as n-tier architecture, is a widely used architecture pattern in software engineering. It divides a software system into multiple layers, where each layer represents a different level of abstraction or functionality. The layers are stacked on top of each other, and each layer only communicates with the adjacent layers, making it easy to modify or replace individual layers without affecting the entire system. This architecture pattern provides a clear separation of concerns and facilitates the implementation of complex systems. The most common layers are the presentation layer, business logic layer, and data storage layer.

Client-Server Architecture:

is a distributed architecture pattern where a client (user interface) interacts with a server to perform a specific function. The client sends requests to the server, and the server processes the requests and sends the results back to the client. This architecture pattern is commonly used in web applications and other network-based systems, where the client can be a web browser or a mobile app, and the server can be a web server or a database server. The main advantage of client-server architecture is its scalability and flexibility, as it allows multiple clients to access the same server concurrently.

Microservices Architecture

is a modern architecture pattern that structures a software system as a collection of small, independent services, each with its own functionality and data storage. Each service communicates with other services using APIs and protocols, making it easy to modify or replace individual services without affecting the entire system. Microservices architecture promotes flexibility, scalability, and resilience in software systems, making it ideal for large, complex applications with changing requirements.

Model-View-Controller (MVC) Architecture

pattern for building user interfaces, web applications, and other interactive systems. It divides the application into three components: the model, the view, and the controller. The model represents the application’s data and business logic, the view represents the user interface, and the controller manages the communication between the model and the view. This architecture pattern promotes separation of concerns and modularity, making it easy to modify or replace individual components without affecting the entire system.

Event-Driven Architecture

is a distributed architecture pattern where software components communicate with each other through events, such as messages, signals, or notifications. The components are decoupled and only communicate with each other through events, which are generated and consumed asynchronously. This architecture pattern promotes scalability, flexibility, and responsiveness in software systems, making it ideal for complex, event-driven systems such as real-time applications, streaming services, and messaging systems.

How to Decide ?

Selecting the appropriate architecture pattern for a software system depends on a variety of factors, including the system’s requirements, constraints, and expected behaviours. Here are some steps you can follow to help you choose the best architecture pattern for your project:

Requirements and constraints

Understanding the system’s requirements is essential in selecting an appropriate architecture pattern. Identify the system’s functional requirements, such as the features, use cases, and workflows, as well as non-functional requirements such as performance, scalability, security, and maintainability.

Domain and problem complexity

Consider the complexity of the problem domain and the overall architecture. For example, a microservices architecture may be suitable for a large, complex, and distributed system, while a simpler monolithic architecture might be more suitable for a small project.

Team expertise

Evaluate your team’s experience and familiarity with the potential architecture patterns. A pattern that your team is already comfortable with may lead to faster development and fewer mistakes.

Integration with existing systems

Consider how the new system will integrate with other systems and components, and select a pattern that will facilitate smooth integration

Scalability and performance

Evaluate how well the architecture pattern will support the expected load, growth, and performance requirements of the system.

Testability

Ensure the selected pattern allows for easy and thorough testing of the system’s components.

Summary

References

Setting things Up

Sun, 19 Mar 2023 15:00:00 +0530

Algorithmic trading, also known as algo trading, is the use of computer programs to follow a defined set of instructions for placing trades. It is a form of automated trading that allows traders to execute orders with speed and precision. Algo trading helps traders to identify and react to market opportunities faster, reduces emotions, and increases discipline. It also allows traders to track multiple accounts and multiple markets simultaneously.

In this post/video we will get started with foundational code in python and cover the following introductory material.

• Setting up yfinance library

• Basic Operations and Plotting

• SMA, EMA, Bollinger Bands

• Simple Strategy based on SMA

Python is a popular language for algorithmic trading. It has a wide range of libraries and tools for performing data analysis, backtesting and paper trading. It is also used for developing trading strategies and optimising them for different markets. Python has a wide range of libraries for performing data analysis, backtesting and paper trading. The most popular libraries include Pandas, Numpy, Scipy, Matplotlib, Scikit-learn, and Statsmodels.

Code

youtu.be/uvby-fWe8…

Definitions

Simple Moving Average : A simple moving average (SMA) calculates the average of a selected range of prices, usually closing prices, by the number of periods in that range.

• www.investopedia.com/terms/s/s…

Exponential Moving Average : An exponential moving average (EMA) is a type of moving average (MA) that places a greater weight and significance on the most recent data points. The exponential moving average is also referred to as the exponentially weighted moving average. An exponentially weighted moving average reacts more significantly to recent price changes than a simple moving average simple moving average (SMA), which applies an equal weight to all observations in the period.

• www.investopedia.com/terms/e/e…

Bollinger Band : A Bollinger Band® is a technical analysis tool defined by a set of trendlines. They are plotted as two standard deviations, both positively and negatively, away from a simple moving average (SMA) of a security’s price and can be adjusted to user preferences.

• www.investopedia.com/terms/b/b…

To get started with algorithmic trading, one needs to understand the different strategies and how to implement them. Strategies include mean reversion, momentum, arbitrage, market making, pair trading, and volatility trading. Mean reversion is the theory that suggests that prices tend to revert to their mean or average. Momentum is the theory that suggests that prices move in the same direction for a period of time. Arbitrage is the strategy of taking advantage of differences in prices of the same security in different markets. Market making is a strategy of providing liquidity to the market by buying and selling the same security. Pair trading is the strategy of buying and selling two correlated securities. Volatility trading is the strategy of trading on the volatility of the market.

Why should you draw Software Architectures ?

Sat, 18 Mar 2023 09:27:30 +0530

Thoughts & Experiences for Drawing Software Architecture

Most Architects represent their thoughts in the form of diagrams. These are of varied types and formats.

Some like to represent in the form of mixed formats representing Data flow, Applications, and Infrastructure all in a single view while others are often selective about what is captured
Some will focus on the direction of arrows, strength(thickness) of lines, and color of lines while others might doodle away
Choosing the amount of detail presented is a choice that is individual too unless stronger standards exist

All of them are correct and there’s always more than a single diagram that can communicate. But why are diagrams so effective in describing architectures? The answer lies in a definition or more so understanding of the Term.

While there are many I love the definition by Mr. Martin Fowler, “the shared understanding that the expert developers have of the system design” ..! In a complex system with multiple services and a varied tech stack, this definition helps intuitively understand why there is a need to draw.

Softwares evolve and are constantly under change. With this, there is a natural need to understand the impact of a change on the overall system. This is critical to maintaining the stability and continuity of the system. How many times has it happened that we find ourselves changing a piece scared to death about it causing a cascading impact in production, or blowing off a downstream system due to a lack of awareness of the overall picture? This awareness is what the Architecture diagram brings in.

So next time you have a change ensure that the first activity within your sprint is to assess the architecture impact. Even if there is none it is important to capture a simple ADR (Architecture decision record) describing the requirement and capturing the conversation.

Best Practises out of the personal experience

Use colors to represent changing states
Ensure that arrows follow a single convention (Data flow, Call Flow, etc)
Group multiple interfaces and move them to a different diagram basis the interoperability method (All external REST APIs are grouped under a single group)
Different environments can have different diagrams with a graduation process guiding them
Limit the details (Focus on requirements and eliminate the noise)

In the next post, I will try to take a sample app and draw it as I think might be good. Stay tuned.

Introduction to Algo Trading

Fri, 17 Mar 2023 13:52:27 +0530

Introduction

In this blog post, we will explore the basics of algo trading and its components. We’ll look at what a ticker is and the difference between quants and technical and fundamental analysis. We’ll also look at the basics of Python, strategies, libraries, backtesting, and paper trades.

https://www.youtube.com/watch?v=uvby-fWe8Ts

Ticker, Liquidity & Order Book

Let’s start by looking at what a ticker is. A ticker is the state of an order book in any given instance. The tick includes the trades executed in the given instance with the bid/ask information. For each trading interval, four price points are available: open, high, low, and close. There are two kinds of participants: buyers and sellers. Participants place orders as bid and ask.

Liquidity can be defined as the flow of bid and ask orders in a given time interval. If the liquidity is high, it means it is easy to trade in the instrument and the slippage cost will be lower for the participants. The details of the bid and ask and trades at any given time are considered as an order book. The rule of the order book is, that the highest bid price in the market is the best bid and the lowest ask is the best ask. Most of the exchanges follow the price-time priority rule, where the bigger bid and smaller ask will be given precedence in trade execution. If the price is the same for the two orders then the time will factor will be used. The order that arrived earlier will be given priority in trade execution.

Order Types

There are different types of orders available, such as market orders, limit orders, stop orders, iceberg orders, fill or kill orders, and immediate or cancel orders.

Market orders execute at the current best price available, while limit orders are placed to buy or sell a security at a specific price.
Stop orders are also alternatively called stop-loss orders, which is an order to buy or sell an instrument once the price of the instrument reaches a specified price, known as the stop price.
Iceberg orders are an order type that slices orders of larger quantity (or value) into smaller orders, where each small order, or leg, is sent to the exchange only after the previous order is filled.
Fill or kill (FOK) orders are an order to buy or sell a security that must be filled in its entirety or else canceled.
Immediate or cancel orders (IOC) are an order to buy or sell a security that attempts to execute all or part immediately and then cancels any unfilled portion of the order.

Quants, Technical & Fundamental Analysis

Now let’s look at the difference between quants and technical and fundamental analysis.

• Quants use mathematical and statistical models to analyze financial data and make predictions.

• Technical analysis is the study of past price movements and trading volumes to identify patterns and make predictions about future price movements.

• Fundamental analysis is the study of economic and financial factors that can affect the price of a security.

Python is a programming language used by quants and traders to develop algo trading systems. It has libraries for data analysis, machine learning, and quantitative analysis.

Strategies are the core of algo trading.

They are the set of rules and conditions that define when and how to enter and exit a trade.

Backtesting is the process of testing a strategy on historical data to evaluate its performance. Paper trading is the practice of simulating the trading of securities without using actual money.

Summary

In conclusion, algo trading is a form of automated trading that allows traders to execute orders with speed and precision. It helps traders to identify and react to market opportunities faster, reduces emotions, and increases discipline.

Git states

Tue, 14 Mar 2023 16:58:31 +0530

Broadly speaking, there are three states in which your files can be: modified, staged and committed. It is crucial to understand these terms, which will help in familiarizing yourself with the git environment

Modified: This refers to the state when you have made changes to the file but you have not committed the changes yet.
Staged: It means that you are ready to commit the changes you have made in your file, marked as modified.
Committed: It indicates that the data is stored in your local database.

Starting a new project with git

Let’s Begin by Creating a Local Repository (Execute the command in a Cell) and simultaneously see how to “commit”, which is creating a snapshot of the entire state of the project at a moment. Github helps you create a remote repository.

Random trivia: Github has recently changed the name of “master” branch to “main” due to cultural sensitivity around slavery.

Git init

output

Initialised empty Git repository in D:/git_training/.git/

The Exclamation Mark ensures that the command is executed as a Terminal Command. The output shows that we have created a local Empty git repository.

Git status

We should be able to evaluate the status of the created repository using the Status command

Git status

This would allow us to check the current status of our Repository at any point in time.

Let’s add a New File to the Folder called hello.txt which has a sample text inside it.

Execute the following to see if Git has started tracking changes.

Git Status

The Output should be something similar to this.

This is very informative. Let us evaluate each of the messages before proceeding further.

On branch master: It means that we are currently working on the master branch
No commits yet: We have not made any commits to the repository so far
Untracked files: This shows that git was able to find some files which are part of the folder that git is trying to track

Git Branch

A branch in Git is simply a lightweight movable pointer to one of these commits. The default branch name in Git is master (now main). As you initially make commits, you’re given a master branch that points to the last commit you made. Every time you commit, it moves forward automatically.

Git Commit

Git Commit - Commits are created with the git commit command to capture the state of a project at that point in time. The most recent state of the project is called HEAD.

Effectively we are trying to achieve the following workflow:

We will now add the files that are currently untracked.

Git add .  Git status

We will be adding the un-tracked listed above in our repository. Notice the .; at the end. This is to select all the files. Alternatively, file names can be specified.

The status has not changed to No Commits which specifies that changes are yet to be committed to the branch master.

Before we start committing Git needs to know who we are in order to maintain a name and email ID along with the commits. Execute the following before making a commit. This is a One Time Activity.

git config --global user.email "you@example.com"
git config --global user.name "Your Name"```

Now, let's Commit

Git commit -m "Initial Commit"

The hyphen m specifies the comment that should be added to commit. Consider it like a changelog that a user can maintain.

The following message confirms that 1 file was changed with 2 lines added into the Git version system. Furthermore, we can track Git Status which we have already learnt.

Working with remote repositories

Github allows you to create remote repositories which can move the project from local to a remote repository.

Git push

After you are done committing your code, it is time to populate your remote repository on Github. This can be done with the Push command on git. If you are working on the master branch and pushing code to the origin repository, then:

git push origin master
Git pull

An easy way to copy an entire repository on a remote repository is to use git clone with the link to the repository on Github. If you would like to download content from a remote repository and match your remote repository to it, you can use git pull.

git pull origin master

You would be accustomed to pulling requests while working in teams. This helps to ensure that contribution by a member is done without affecting the entire workflow and it also acts as a system of the check so that the original repository is not negatively impacted.