Scalability in web applications has become crucial in the past years. An ever-growing number of devices and clients, connected to the Internet, makes it difficult to handle the traffic by a single server. Scalability is a characteristic of a system that tells if expanding the resources of a server(s) will make it run faster (or handle more load). Not all applications, systems, and networks are scalable by default, they must be designed in a certain way to support scalability. Cloud solutions allow large-scale computing by definition, and IoT platforms should support that as well. When dealing with multiple devices connected to the solution, the platform must stay reliable to manage, store, transform, and analyze huge amounts of data. In the modern world, where everything is connected, an IoT platform becomes a critical point. Every connection causes a reaction – it could be a simple request for current data, but also a more sophisticated query to calculate yearly MTBF (Mean time between failures).

IIoT (Industrial IoT) supports hundreds of thousands of connected devices but also allows to see them as a group as line, shop floor, or an entire factory. Enabling new lines, replacing devices, and expanding the factories should be fast and easy. However, scaling down is also important – switching off the shop floor for energy saving or to reduce supply when demand is reduced is what IoT platform must be able to do. And it must be done seamlessly.

Horizontal Scaling

Clustered environments expand by adding new servers. That’s the horizontal scaling – resources of a system are increased along with new nodes. On the other hand, there is a vertical scaling where resources are added to the single node. It is like a dilemma of whether it’s the number or the size that matters.

Vertical Scalability

Scaling vertically (also referred to as up/down) means to add or remove resources within a single server. These resources are CPU cores, RAM, or storage. In this case, the server gets better throughput. However, in a monolithic environment (as opposed to clustered one), when a server goes down, the whole platform might not be accessible, which means it causes downtime. The next aspect is that to add resources the server must be turned off, which means downtime once again. The other drawback is that scaling up has its upper limit. It is not possible to add RAM or CPU indefinitely.

Horizontal Scaling

Advantages

  • No need to change the application code. The same code will immediately run on the server with new resources
  • There is only one server to manage, which makes things a lot simpler and performance boost is easy to see.
  • Might be cheaper in terms of manageability and administration

Disadvantages

  • Downtimes during hardware changes
  • High-end hardware may be expensive
  • There is an upper limit, especially when using Cloud pre-defined instances

Horizontal Scalability

Scaling horizontally (also referred to as out/in) means the new nodes are added to the cluster. There is possibly no limit since each new node has its own resources. Making the system more powerful by adding new nodes involves only the cluster manager. This solution is not prone to downtimes. New nodes are added seamlessly, and failing nodes do not put the whole system down. Besides, non-functional server nodes can be easily replaced.

The great thing about horizontal scaling is that it automatically introduces High Availability. This scaling approach can only happen in the clustered infrastructure. Each node has exactly the same settings and application running, making it possible to take over the traffic of another server that went down, either by a user, system, or application error.

Horizontal Scaling

Advantages

  • Servers are cheaper as nodes are smaller
  • Easy to put up fallback plan, as replacing failing nodes is easy
  • Scaling is seamless and does not introduce downtime – it is easy to scale out when nodes are not needed.
  • Theoretically no limit in scaling

Disadvantages

  • Code is more complex which leads to more errors
  • Licensing might be more expansive as there are more nodes to license
  • Higher complexity in terms of manageability and administration

Active-Active cluster in ThingWorx 9

The scaling opportunity was introduced in ThingWorx 9. High availability and Active-Passive clusters were supported before, but the new version allows to easily scale out/in the ThingWorx platform. In version 9 the ThingWorx cluster can work in Active-Active mode. That means all nodes are taking an active part in the whole solution. This time, every node can handle the traffic – the whole system has more capacity making it possible to connect even more devices. IoT platform needs to be resilient, fit into any use case, handle the peaks of network traffic, optimize the costs and minimize downtimes by avoiding a single point of failure.

With the Active-Passive approach, only one server is live. It performs all processing and maintains the live connections to other systems. In parallel, there is a second passive instance that is updated regularly and shares exactly the same configuration. It is often referred to as a mirror instance. It does not maintain active connections but it will become active once the master server fails.

The new approach gives even higher availability by making all ThingWorx servers active. Data is synchronized between all servers that maintain live connections to other systems and handle the web traffic from clients and devices. Data is being processed in parallel resulting in a cluster that can process more data than a single instance. Moreover, the new Active-Active cluster is scalable horizontally, making it easy to add new nodes to simply increase processing power, or reduce the number of nodes if such processing power is no longer needed. Now, it is possible to manage large amounts of IIoT data at scale.

What is the Active-Active cluster made of?

ThingWorx uses Apache Ignite for data synchronization between nodes. This powerful tool makes the cluster flexible and easy to maintain. Apache Ignite can act as:

  • In-memory cache, a powerful, low-latency, high-performance Key-Value cache (also supporting ANSI SQL)
  • In-memory Data Grid, an advanced data grid on top of RDBMS allowing to gain 100x acceleration when accessing data
  • In-memory Database, a multi-tier scalable out and up across RAM, NVRAM, Flash, and disk

For ThingWorx cluster data must be available for every node immediately. With a load balancer in place, the node which gets the metering data from a device must save that to persistent storage (e.g. InfluxDB) but the same update must be available for other nodes. Thus the Apache Ignite in-memory data caching is the best choice. Data is being synchronized with an underlying database using an additional in-memory layer.

Horizontal scalability in ThingWorx 9

The great thing about Apache Ignite is that it is scalable by default. There is no need to have any additional software, new nodes will be added automatically to the cluster. It is done thanks to Service Discovery – this bit of software makes it possible for every node to discover each other. In ThingWorx 9, Ignite uses Zookeeper Discovery which makes it possible to scale to 100s or 1000s nodes. For smaller clusters, it is advised to use TCP/IP Discovery which is the default for Apache Ignite.

To summarize, the Active-Active ThingWorx 9 cluster consists of:

  • Apache Zookeeper to manage and enable service discovery
  • Apache Ignite to synchronize data between nodes and speed up database queries
  • Multiple ThingWorx platform instances running simultaneously

Horizontal scalability in ThingWorx 9

The diagram shows that all ThingWorx instances are live and actively maintain the connections. There is a load balancer that knows exactly how many requests each node has handled and to which node to redirect a new request, depending on the selected balancing algorithm. Everything is managed by Apache Zookeeper, which is also used as a discovery service for Apache Ignite instances. Ignite’s cache is partitioned (by default, but that is configurable) between all nodes. Ignite also allows creating a Compute Grid, which helps with splitting up the tasks between all nodes, making processing faster, and allowing to analyze more data at a time. New opportunities for calculating KPIs and showing them live on the dashboard is just a matter of parallelizing existing algorithms.

Conclusion

Both scaling approaches have pros and cons, and the choice depends on specific use cases. However, most often horizontal scaling is a better option. For the ThingWorx platform, it is a major step forward to support Active-Active clustering. Multiple servers running the same application enable new opportunities in data processing, analysis, and transformation. IIoT solutions that depend on hundreds of thousands of devices can now be implemented with a much better estimation of cost and effort, even higher availability and reliability. It allows saving money while reducing unexpected downtimes.

ThingWorx 9 has become an enterprise platform allowing to easily scale the whole solution out and in. During a holiday season or sports event, the platform will remain reliable even with unexpected peaks in client requests or amount of devices connected. Companies can benefit from horizontal scaling because it helps expand factories with new lines or shop floors, even more, connecting entire factories. High availability will benefit in reducing costs by eliminating unexpected downtimes. That makes it possible to focus on new opportunities instead of planning the fallback scenarios. More processing power allows for more sophisticated machighhine learning models and that opens up new horizons for helping companies to be one step ahead.

If you need help with implementing new or upgrading existing solutions based on ThingWorx, please contact us.

_All posts in this category

blogpost
Articles

Lesson Learned Explained: Digitalization of reporting processes in the glass packaging manufacturing industry

A client, a leader in the glass packaging manufacturing sector, identified the need to implement an integrated production data management system to replace outdated, manual reporting methods.

Read more
blogpost
Articles

Lesson Learned Explained: Improving KPIs in the FMCG Industry through automation and data analysis on semi-automated production lines

Introduction In the highly competitive food and beverage industry, achieving optimal Key Performance Indicators (KPIs) such as availability, performance, and quality is essential for maximizing operational efficiency and profitability. A client operating semi-automated production lines was experiencing persistent underperformance in these KPIs. To address this issue, the company required a robust and precise data-driven approach […]

Read more
blogpost
Articles

Lesson Learned Explained: how proper data collection and storage proved crucial in predictive maintenance

In the aerospace and defense industry, which is characterized by particularly high requirements for precision and reliability, key performance indicators in maintenance, failure prediction or machine condition monitoring, are crucial.

Read more
blogpost
Articles

Industry 4.0 in the context of manufacturing companies

Industry 4.0, also referred to as the fourth industrial revolution, is a concept encompassing a complex process of technological and organizational transformation of companies, which began in 2013.

Read more
blogpost
Articles

OEE: is your company stuck in a manipulation trap?

If you think OEE has no secrets to you and your plant maintain highest OEE results… think again. Harsh truth is that most manufacturing plants’ OEE land somewhere between 35 and 43%. They just don’t know about that.

Read more
blogpost
Articles

How to increase production efficiency without investments in the shop floor?

You don't have to replace your machines with the new ones to make your production "smarter" and more efficient. Your shop floor is a data mine, and digitization is the key to unlock its value.

Read more
blogpost
Articles

Navigating ThingWorx: Expert Solutions for IoT Challenges

Whether you're a seasoned user or new to ThingWorx, overcoming these hurdles is crucial for IoT success. In this blog, we'll uncover the top challenges, offer tips for newcomers, discuss strategies to enhance your ThingWorx experience, and explore the importance of upgrading your ThingWorx solution.

Read more
blogpost
Articles

5 ways Manufacturers Can reduce energy Costs 

Very often overlooked or accepted as-is due to being an integral part of the production – in times of crisis and cost-cutting, it can become a major factor that improves the bottom line of your organization. Where to look for significant energy savings to reduce energy costs? How manufacturers should focus on these goals to make an impact? 

Read more
blogpost
Articles

5 steps to increase energy efficiency for Manufacturers

According to the German Federal Statistical Office, energy consumption by the industry for the production of goods hardly changed between 1995 and 2019. In 2019, it amounted to 3,874 petajoules. This means that it remains at the same level as in previous years. Germany plans to halve its primary consumption of energy by 2050, with the industry having to take part. In the era of the energy crisis, the drive to improve corporate energy efficiency is entering a new level.

Read more
blogpost
Articles

7 ways how data visibility helps manufacturing improve efficiency

In the manufacturing industry, efficiency is key to staying competitive and profitable. One way to improve efficiency is through data visibility. By having access to real-time visibility of the operational data throughout the manufacturing process, companies can identify bottlenecks, monitor production lines, and make data-driven decisions. In this article, we'll explore how data visibility can help manufacturing companies improve their efficiency and ultimately their bottom line.

Read more
blogpost
Articles

Energy Advisor for Manufacturing – energy savings for industry

Energy consumption bills have been a major focus in virtually all areas of industry for many years, and the current global situation is a factor driving up demand for solutions that make it easier to monitor energy consumption or save energy. In an industrial environment, where the cost of utilities has always been one of the main business drivers, the latest twists and turns in the global economy: inflation, natural resources supply challenges and the threat of fluctuating and unpredictable increases in energy costs – are arguably the most important economic factors these days.

Read more
blogpost
Articles

Digital Transformation – technological trends for the successful evolution of the automotive industry

The new opportunities associated with Industry 4.0 require companies to adapt to the new environment. In this article, we take a closer look at the current situation in the automotive industry, discuss the challenges it faces and the opportunities offered by digital transformation.

Read more
blogpost
Articles

Here’s why you should use Azure Cloud for your IoT Solutions

Digital transformation is reshaping the way how people, products, assets, data, and operations are connected to create an outcome for the customers and switch digital data into the customer experience. Digital technologies create new business opportunities across the globe and enable the evolution of enterprise, products and processes for valuable capabilities. The article describes the Internet of Things concept and shows how to use cloud infrastructure to take advantage of the digital transformation in business.

Read more
blogpost
Articles

IoT Data Visualization

Internet of Things systems have one huge advantage – they can collect lots of data. Temperature, items produced, amount of remaining liquids, humidity, machine uptime, damaged products, vehicle routes, electricity used. Based on this, we can get even more value by calculating KPIs to indicate how close to perfect our system is. Going manually through an endless stream of numbers changing every couple of hours is impossible and inefficient. The simple solution is to visualize the data by using a carefully selected chart type, that will exponentially increase its value. In this article, I want to show you the benefits of good data visualization and explain how it can be applied in your solution.

Read more
blogpost
Articles

Coronavirus & production continuity – IoT & AR for industry

The sudden epidemic of SARS-CoV-2 coronavirus and the COVID-19 disease has affected every sphere of the modern world. Caused by an epidemiological threat, the restriction of the flow of people, goods and services has already negatively affected the continuity of production processes in many industries, and each day this situation persists, without credible forecasts of a return to normality, will pull others into the spiral of detention threats.

Read more
blogpost
Articles

High Availability in IoT applications

Modern web applications that serve data and content to millions of users run in clustered environments. Dealing with a huge amount of connections at the same time requires many resources. That resources are CPU time and RAM. Each and every connection produces an additional load on the server. Having just one server would require a vast amount of resources and we already know that Moore's Law is no longer valid. That said we need multiple servers ready to respond to users’ requests. Web applications use HTTP(S) protocol which works in the request-response communication method. Upon user's request, the server will prepare the response and send it back. Every response needs to be processed by the server and that's the key factor in provisioning resources for the server - requests may be different, starting from simple static data like CSS or images to more demanding like requesting data from database, filtering or calculating certain data.

Read more
blogpost
Articles

IoT Hub Connector for ThingWorx – Connecting ThingWorx Platform with The Azure Cloud

The ThingWorx IoT Hub Connector is the bridge that connects the ThingWorx platform to the Azure cloud (more specifically to Azure IoT Hub). It provides secure (certificate-based) communication, thanks to which we can use the enormous possibilities of cloud computing. It is also a very extensive, but generic solution.

Read more
blogpost
Articles

What is AIoT? Artificial Intelligence of Things in Industry 4.0

Synergy, this is how you can define this natural combination of technologies that will significantly affect the appearance of the future of production. Recently, in addition to the Internet of Things (IoT), artificial intelligence (AI) is gaining more and more popularity. The connection of these two technological trends heralds a real revolution in industry 4.0.

Read more
blogpost
Articles

What’s new in ThingWorx 9.0?

This year's Liveworx, due to the global situation caused by the COVID-19 pandemic, has exceptionally been held only online. On the very first day it was announced that a new version of ThingWorx - the most popular platform maximizing the potential of the industrial Internet of Things - will be released for commercial use. According to the announcements of Joe Biron, CTO responsible for IoT technology at PTC, it is supposed to be faster, cheaper and more effective.

Read more
blogpost
Articles

Machine Learning & ThingWorx vs COVID-19

The evolution of artificial intelligence, as well as machine learning has gained momentum in the last few years. The constant drive to increase the computing power of computers has given the opportunity to use algorithms and mathematical apparatus created in the second half of the 20th century on an unprecedented scale. With the imminent arrival of a commercial quantum computer, we can slowly become afraid of scenes from the movies with Arnold Schwarzenegger as Terminator. However, without going too far into the future, and focusing on the present, I decided to check the possibilities offered by the technology available today and whether it can be used to predict the spread of COVID-19 disease.

Read more
blogpost
Articles

5G network – the key to the Industrial Internet of Things

The Industrial Internet of Things changes our view on the classic concept of production today. The largest manufacturing companies are eager to reach for technological solutions that streamline and increase the efficiency of production processes. With the development of subsequent technologies, interest in the concept of industry 4.0 increases, and thus, the number of connected devices. To meet the demand for infrastructure with adequate capacity and speed it was necessary to evolve towards the next generation of networks - 5G. How will 5G affect the development of Factories of the future and why is this crucial from the perspective of the Internet of Things? You will learn everything from the article.

Read more
blogpost
Articles

How is the Internet of Things changing the aviation industry?

The Internet of Things is being used more and more, and even global giants are already using the optimization or information gathering solutions. All this to improve existing processes, acquire, process and report more data, or even introduce new services for their clients, thereby increasing sales revenue. How is the Internet of Things changing the reality of the aviation industry?

Read more
blogpost
Articles

Rockwell Automation TechED EMEA 2019 shows that Industry 4.0 is closer than you think

Following up our partnership with Rockwell Automation, our team of business and technical experts found their way to Munich, Germany to participate in TechED EMEA. It is a unique event during which you can listen, talk and test the latest news in the field of industrial automation and directly participate in the next industrial revolution.

Read more
blogpost
Articles

Internet of Things and its impact on Automotive Industry

Nobody would be surprised by a statement that the Automotive industry is innovative in its nature. Cars are machines that historically revolutionized many areas of industry themselves, and today we can’t help but notice that the industry has not slowed down even for a moment and has always been focused on technological growth that serves both the business and its customers.

Read more
blogpost
Articles

Internet of Things in logistics

It might seem that the Internet of Things is dedicated only to industry. However, transport and logistics are closely connected to it, and they can also take advantage of the ongoing revolution. In the end, every industry uses transport services or logistic processes to some extent.

Read more
blogpost
Articles

LiveWorx 2019 – IoT trendsetting for next year

As every time for the last couple of years, our strong team of representatives was present in Boston to attend one of the most important IoT events of the year – LiveWorx, 2019 edition.   Get the ball rollin’   And even with 10 participants from various business areas, actively seeking new information and presentations, […]

Read more
blogpost
Articles

The fourth Industrial (R)evolution

Industry 4.0 and its foundation, the Internet of Things (IoT – especially in industrial version: IIoT), in the last years have dominated the way of thinking of advanced management concerning manufacturing business resources.

Read more
blogpost
Articles

How is ThingWorx different from Axeda and should you consider moving?

PTC is phasing out its Axeda platform. Although they look similar at first sight, Axeda and ThingWorx differ very much in capabilities.

Read more
blogpost
Articles

Are your data sources ready? InfluxDB support of ThingWorx

While ago, PTC announced a new release of ThingWorx Platform (8.4). Among other exciting features, this release has OOTB support for integration with InfluxDB.

Read more
blogpost
Articles

The Internet of Things – 9 trends for 2019!

The approach that has been underestimated for a long time is nowadays spreading its wings and becoming a crucial set of solutions for global corporations. However, not only for them as medium and small companies are following the biggest ones as well. Thus, the advantages of the Internet of Things have been recognized by everyone. […]

Read more
blogpost
Articles

Is now the time to invest in IoT?

Recent years were hyped about the Internet of Things (IoT) and how it was supposed to re-shape the whole industry. However, so far we’ve seen many more people talking about it than actually implementing it enterprise-wide. Is IoT mature enough to be “the real deal” and bring measurable value to businesses?   The term IoT […]

Read more

Let’s get in touch

Contact us