Python on CafeIO

Deep Dive into Architecture Patterns, Part 2

Sun, 09 Apr 2023 09:00:00 +0530

Deep Dive into Architecture Patterns. This Post is focused on understanding different technology stacks and their usage.

This might very well be tangential to the idea of patterns, but I feel it is significant to be aware of the different technology stacks and from the core idea of patterns.

A technology stack refers to a set of technologies that are used together to create a software application or system. Different technology stacks are typically named after the key technologies that they include, such as the operating system, programming language, web server, and database.

Tech Stack Management

Some Popular Technology Stacks

Here’s a high-level view on some popular technology choices that we are currently presented with.

LAMP Stack : Linux: LAMP stack is based on the Linux operating system, which is an open-source operating system that is widely used for servers and desktops. Apache: Apache is a free, open-source web server software that is used to deliver web content over the internet. MySQL: MySQL is a popular open-source relational database management system (RDBMS) that is used to store and retrieve data. PHP: PHP is a popular open-source server-side scripting language that is used to create dynamic web pages and web applications.
MEAN Stack : MongoDB: MongoDB is a NoSQL document-oriented database that is used to store data in a flexible, JSON-like format. Express.js: Express.js is a lightweight web application framework that is used to build web applications and APIs in Node.js. AngularJS: AngularJS is a JavaScript-based front-end web application framework that is used to build dynamic and responsive user interfaces. Node.js: Node.js is a server-side JavaScript runtime environment that is used to run JavaScript on the server-side.
MERN Stack : MongoDB: Same as MEAN stack, MongoDB is used as the database in MERN stack.Express.js: Same as MEAN stack, Express.js is used as the server-side framework.React.js: React.js is a popular front-end JavaScript library that is used to build user interfaces.Node.js: Same as MEAN stack, Node.js is used as the server-side runtime environment.
MERN with Next.js Stack : MongoDB: Same as MERN stack, MongoDB is used as the database in this stack.Express.js: Same as MERN stack, Express.js is used as the server-side framework.React.js: Same as MERN stack, React.js is used as the front-end JavaScript library. Node.js: Same as the MERN stack, Node.js is used as the server-side runtime environment.Next.js: Next.js is a popular React-based framework that is used to build server-side rendered web applications.
.NET Stack : .NET Framework or .NET Core: .NET Framework is a free, open-source development platform for building Windows-based desktop and web applications. .NET Core is a newer version of the framework that is designed to be cross-platform.C#: C# is a modern, object-oriented programming language that is used to develop applications on the .NET platform. Visual Studio IDE: Visual Studio is an integrated development environment (IDE) that is used to develop, test, and deploy .NET applications.SQL Server: SQL Server is a popular relational database management system (RDBMS) developed by Microsoft.
Python Flask Stack : Python: Python is a high-level programming language that is used for web development, data analysis, and artificial intelligence. Flask: Flask is a lightweight web application framework that is used to build web applications and APIs in Python. PostgreSQL: PostgreSQL is a popular open-source relational database management system (RDBMS) that is used to store and retrieve data.
Django Stack : Django: Django is a high-level Python web framework that is used to build web applications and APIs quickly and efficiently. PostgreSQL: PostgreSQL is a popular open-source relational database management system (RDBMS) that is used to store and retrieve data. Nginx: Nginx is a high-performance web server that is used to deliver web content over the internet.
Java Stack : Java: Java is a widely used programming language that is used to develop a wide range of applications, from desktop and mobile apps to web and enterprise applications.Spring Framework: Spring is a popular open-source framework that is used to build web applications and enterprise-level software systems. Hibernate: Hibernate is an object-relational mapping (ORM) framework that is used to manage database operations in Java-based applications. Apache Tomcat: Apache Tomcat is a popular open-source web server and servlet container that is used to deploy and run Java-based web applications.
Serverless Stack : AWS Lambda or Azure Functions: Serverless stack is a technology stack that doesn’t require any server management or infrastructure. AWS Lambda and Azure Functions are popular serverless computing services offered by Amazon Web Services and Microsoft Azure, respectively._ Amazon S3 or Azure Blob Storage_: Amazon S3 and Azure Blob Storage are popular object storage services that can be used to store and retrieve data in a serverless environment._ Amazon API Gateway or Azure API Management:_ Amazon API Gateway and Azure API Management are popular API gateway services that can be used to manage and secure APIs in a serverless environment.

Factors to keep in mind before selecting Tech Stack

When selecting a technology stack, there are several factors that need to be considered. Here are some key factors that should be kept in mind:

Project requirements: Consider the requirements of the project, including the type of application, the features it needs to support, and the expected usage. The technology stack should be able to support these requirements and provide the necessary functionality.
Scalability: Consider the scalability requirements of the project, including the expected growth of the application over time. The technology stack should be able to scale to meet the demands of the application without sacrificing performance.
Cost: Consider the budget for the project and the cost of the technology stack. Some technology stacks may require more expensive hardware or licensing fees, while others may be more cost-effective.
Development resources: Consider the skillset and experience of the development team. The technology stack should align with the team’s expertise to ensure efficient development and maintenance.
Community support: Consider the level of community support for the technology stack. A strong community can provide valuable resources, tools, and support for developers, as well as ensure the longevity and sustainability of the technology.
Integration: Consider the integration requirements of the project. The technology stack should be able to integrate with other systems and tools as needed.
Security: Consider the security requirements of the project, including data privacy, regulatory compliance, and potential vulnerabilities. The technology stack should provide robust security features and be regularly updated to address new security threats.

Overall, it is important to carefully evaluate the specific requirements and constraints of the project to select the most appropriate technology stack.

Some popular technology stacks mentioned are LAMP, MEAN, MERN, MERN with Next.js, .NET, Python Flask, Django, Java, and Serverless stack. Factors to consider when choosing a technology stack include project requirements, scalability, cost, and development resources.

Deployment Patterns

Deployment Approaches

The deployment options available for a technology stack depend on the specific components of the stack and the requirements of the project. However, here are some common deployment options that are available for most technology stacks:

On-premises deployment: This is a traditional deployment method where the application is deployed on hardware that is located on-site or within the organization’s premises. This deployment method requires the organization to manage the hardware, software, and security of the application.
Virtual machines (VM) deployment: In this deployment method, the application is deployed on a virtual machine that is hosted on a public cloud or private infrastructure. This method provides greater flexibility than on-premises deployment and allows the organization to scale resources up or down as needed.
Container deployment: Container deployment involves deploying the application in a container, such as Docker, which encapsulates the application and its dependencies. This method provides greater portability, as the container can be deployed on any infrastructure that supports the container runtime.
Serverless deployment: In a serverless deployment, the application is deployed as a set of functions that run in a serverless computing environment, such as AWS Lambda or Azure Functions. This deployment method eliminates the need to manage the underlying infrastructure and provides greater scalability.
Platform as a Service (PaaS) deployment: PaaS deployment involves deploying the application on a cloud-based platform, such as Google App Engine, Heroku, or Microsoft Azure App Service. The platform manages the underlying infrastructure and provides developers with a pre-configured environment for deploying and running their applications.
Infrastructure as a Service (IaaS) deployment: IaaS deployment involves deploying the application on a cloud-based infrastructure, such as Amazon Web Services (AWS) or Microsoft Azure. This deployment method provides organizations with greater control over the underlying infrastructure and allows them to customize the environment to meet their specific requirements.

Factors Influencing Deployment Strategies

When selecting a deployment method for a technology stack, there are several factors that should be considered. Here are some key choices that should be made:

Scalability: Consider the expected traffic and usage of the application. If the application is expected to experience high levels of traffic or usage, a scalable deployment method such as serverless or container deployment may be more appropriate.
Cost: Consider the budget for the project and the cost of the deployment method. For example, on-premises deployment may be more cost-effective for small projects, while cloud-based deployment may be more cost-effective for larger projects.
Flexibility: Consider the level of flexibility required for the project. For example, if the application needs to be deployed in different environments, a container or serverless deployment may be more appropriate.
Maintenance: Consider the level of maintenance required for the deployment method. For example, on-premises deployment may require more maintenance than cloud-based deployment, which is managed by the cloud provider.
Security: Consider the security requirements for the project. For example, if the application handles sensitive data, a cloud-based deployment method may require additional security measures to ensure the data is protected.
Integration: Consider the integration requirements for the project. For example, if the application needs to integrate with other systems, a platform as a service (PaaS) deployment method may be more appropriate, as it provides pre-configured environments for deploying and running applications.

Overall, it is important to carefully evaluate the specific requirements and constraints of the project to select the most appropriate deployment method for the technology stack.

The section discusses different deployment approaches available for technology stacks, including on-premises, virtual machines, container, serverless, PaaS, and IaaS deployment. It also highlights the factors that should be considered when selecting a deployment method, such as scalability, cost, flexibility, maintenance, security, and integration requirements. The selection of the deployment method should depend on the specific requirements and constraints of the project.

Summary

In summary, we discussed the popular technology stacks such as LAMP, MEAN, MERN, MERN with Next.js, .NET, Python Flask, Django, Java, and Serverless stack. When choosing a technology stack, it is important to consider project requirements, scalability, cost, and development resources.

Deployment approaches are also important when it comes to technology stacks. There are several deployment methods available, including on-premises, virtual machines, container, serverless, PaaS, and IaaS deployment. Each deployment method has its own advantages and disadvantages. For example, serverless deployment eliminates the need to manage underlying infrastructure and provides greater scalability. The selection of the deployment method should depend on the specific requirements and constraints of the project. When selecting a deployment method, scalability, cost, flexibility, maintenance, security, and integration requirements should be considered.

In summary, the selection of the technology stack and deployment method should depend on the specific requirements and constraints of the project. It is important to carefully evaluate the project’s needs and select the appropriate technology stack and deployment method. By doing so, you can ensure that the project is successful and meets its goals

Unlocking The Secrets of Algo Trading: Learn How to Use Python for Getting Data and Strategies…!!

Tue, 04 Apr 2023 09:00:00 +0530

With Python’s limitless libraries and tools, we can build a robust foundation to predict market trends, analyse financial data, and create advanced financial models.

Hey there! In this post, I’m going to talk about setting things up for algo trading. In my previous video, I covered many aspects of algo trading, including technology, tools, markets, and data. I will build on top of that in this blog post. I would like to point out that this post is accompanied by a video which I shared before.

You can watch the supplementary video and Follow Along the Video.

Hardware & Software Setup

You don’t need a fancy computer for this, anything online or a simple computer will work.

8 GB of RAM with an i5 Processor. Something lower might work too.
Apple Silicon (m1/m2) is great too if you can install the required dependencies. There are few libraries which do not work on it.
Anaconda Environment with Python 3.7. Though higher version of python can be taken, however I have found this to be the common denominator which most libraries support.
List of Libraries to Install
- Pandas
- Numpy
- Matplotlib, Seaborn, Plotly
- yfinance
- Cufflinks
Bonus : Just use Google Colab and install yfinance, Cufflinks in there and get started

Here’s what we will focus on

We will be setting up a python environment, getting market data using Yahoo Finance, and doing some basic statistical analysis. We will then create a simple strategy and compare returns.

Get market data from Yahoo Finance.
Play with the data, get data from NSE, and do stats on it.
Calculate moving averages and do plotting.
Try to do something known as Bollinger bands, which is a technical indicator.
Make a simple moving average-based strategy.

More Details

After installing the libraries, we will use Yahoo Finance to get market data. We will define a variable ticker and set start and end dates. Then, we will download the data and create a data frame. We will also reduce the number of days to make it easier to plot.

Next, we will plot the high, lows, and search for patterns. We will also calculate some basic formulas like simple moving average, exponential moving averages, and Bollinger bands. Bollinger bands is a plus-minus two standard deviations up and down from a simple moving average.

Finally, we will make a simple moving average-based strategy. The strategy will be to sell if a condition is met and hold if it is not.

The Complete Code is shared below

Install necessary libraries.
Define variable ticker and set start and end dates.
Download data and create a data frame.
Reduce the number of days to make it easier to plot.
Plot high lows and search for patterns.
Calculate basic formulas like simple moving average, exponential moving averages, and Bollinger bands.

Working with a Strategy

In the world of algorithmic trading, having a well-defined strategy is crucial for success. A trading strategy is simply a set of rules that traders follow to generate signals that indicate when to buy, sell, or do nothing.

One popular trading strategy is using the simple moving average (SMA). The SMA is a technical indicator that helps traders identify trends in the market. It calculates the average price of a security over a specific time period, such as 12 days, by adding up the closing prices of each day and dividing it by the number of days in the period.

When using the SMA, traders typically buy a stock when its closing price is higher than its simple moving average and sell it when the closing price is lower. This approach can help traders identify potential uptrends or downtrends in the market and make informed decisions on when to enter or exit a trade.

However, it’s important to note that no trading strategy is foolproof. Market conditions can change rapidly, and unforeseen events can impact a stock’s performance. Therefore, it’s essential to continually monitor the market and adjust trading strategies as needed.

In addition to the SMA, there are many other technical indicators and trading strategies that traders can use to make informed decisions in the market. Some popular ones include the relative strength index (RSI), the moving average convergence divergence (MACD), and the Bollinger Bands.

When studying financial engineering and data science, it’s essential to clean up data to ensure that the calculations are accurate. This often involves dropping unnecessary columns and removing any errors or outliers that may skew the results.

In summary, having a well-defined trading strategy is essential for success in algorithmic trading. The SMA is just one of many technical indicators and trading strategies that traders can use to make informed decisions in the market. However, it’s important to continually monitor the market and adjust strategies as needed to stay ahead of the game.

Complete Code

#pip install yfinance plotly cufflinks

# Core Imports
import pandas as pd
import numpy as np
import datetime
import yfinance as yf
import matplotlib.pyplot as plt

#Ticker is the stock that we want to get
Ticker = "^NSEI"


# Get Sample data based on a Ticker
end1 = datetime.date.today()
start1 = end1 - pd.Timedelta(days=5)

df = yf.download(Ticker, start=start1, end=end1, interval="5m" )
print(df.head())
df.info()

# Plotting the Data

df1a = df.copy()
df1a.loc['2023-02-13', ['Open', 'High', 'Low', 'Close']].plot(grid=True, linewidth=1, figsize=(14, 9))

# Just the Closing Price
df1a['Close'].plot(grid=True, linewidth=1, figsize=(14, 9))

# Core Calculation Functions 
# SMA
def get_sma(prices, rate):
  return prices.rolling(rate).mean()

# EMA
def get_ema(prices, rate):
  return prices.ewm(span=ema, adjust=False).mean()

# Bollinger Bands

# Bollinger bands are a technical analysis tool used by traders to identify potential entry and exit points in the market. They are created by plotting a moving average of the price along with two standard deviation lines above and below it. By doing so, Bollinger bands can provide an indication of whether a stock is overbought or oversold. They also help traders identify potential breakouts or reversals in the market. With this knowledge, traders can make more informed decisions when entering or exiting a position in the market.

def get_bollinger_bands(prices,rate):
    sma = get_sma(prices, rate)
    std = prices.rolling(rate).std()
    bollinger_up = sma + std * 2 # Calculate top band
    bollinger_down = sma - std * 2 # Calculate bottom band
    return bollinger_up, bollinger_down


# Other Functions

def download_daily_data(ticker, start, end):
    """ 
    The function downloads daily market data to a pandas DataFrame 
    using the 'yfinance' API between the dates specified.
    """
    data = yf.download(ticker, start, end)
    
    return data

def compute_daily_returns(data):
    """ 
    The function computes daily log returns based on the Close prices in the pandas DataFrame
    and stores it in a column  called 'cc_returns'.
    """
    data['cc_returns'] = np.log(data['Close'] / data['Close'].shift(1))
    return data

# Generate Bollinger bands for the above

bollinger_up, bollinger_down = get_bollinger_bands(df1a['Close'],20)


# Plot the Results

symbol = 'NSE'
closing_prices = df1a['Close']

plt.title(symbol + ' Bollinger Bands')
plt.xlabel('Days')
plt.ylabel('Closing Prices')
plt.plot(closing_prices, label='Closing Prices')
plt.plot(bollinger_up, label='Bollinger Up', c='g')
plt.plot(bollinger_down, label='Bollinger Down', c='r')
plt.legend()
plt.show()


# Strategy Perform Calculations

# Remove the Un-necessary Columns
df1a.drop(columns=['High', 'Low', 'Volume'], inplace=True)

# Create a new colum which captures the percentage cahnge from previous day
df1a['cc_returns'] = df1a['Close'].pct_change()

# Define a short 12 day sma 
sma = 12
df1a['sma'] = df1a['Close'].rolling(window=sma).mean()

print(df1a.head())
print(df1a.tail())

# If the closging price is higher that sma buy else do nothing
df1a['position'] = np.where((df1a['Close'] > df1a['sma']), 1, 0)
df1a['position'] = df1a['position'].shift(1)
df1a['position'].value_counts()

# Plotting the above strategy returns

df1a['strategy_returns'] = df1a['cc_returns'] * df1a['position']


df1a['strategy_returns'] = 1 + df1a['strategy_returns']
df1a['cc_returns'] = 1 + df1a['cc_returns']

print(df1a.head())
print(df1a.tail())

df1a[['cc_returns', 'strategy_returns']].cumprod().plot(grid=True, figsize=(9, 5))

print('Buy and hold returns: ', np.round(df1a['cc_returns'].cumprod()[-1], 2))
print('Strategy returns: ', np.round(df1a['strategy_returns'].cumprod()[-1], 2))
HyperWrite Logo