Smart grids fuse energy development with technological advancements. Using sensors, IoT, and other computing devices, there is a provision for two-way communication between consumers and utility providers in a smart grid. As an artificially intelligent system, a huge amount of data comes from various sources, e.g. smart meters. All the unstructured data gathered from these sources can only be valuable with smart grid analytics.
Smart grid analytics are systematic computational analyses of the data produced in the grids. With these analytics, one can get a more precise interpretation, communication, and identification of data trends or meaningful patterns from the data that comes in. Thus, it is essential to improve grid operations and predict the next course of action.
A Brief History
From the 1990s, attempts at electronic metering, control and monitoring evolved into smart grids. From automated meter readings in the 1980s to Advanced Metering Infrastructures in the 1990s, attempts have been made to go beyond measuring power usage to maximizing the information.
The concepts of analytics can be traced back to the 19th century with Frederick Winslow Taylor’s time management exercises and Henry Ford’s measurements of assembly lines’ speeds. It would interest you to know that predictive analytics (which is now of high importance in smart grids) started in the 1940s. However, it did not attract any attention until the 1960s, when decision support systems became popular. By 2005, businesses applied analytics to make iterative explorations on past activities and make decisions to plan the future.
Applying analytics to smart grid data is what birthed smart grid analytics. The problem of big data (as Roger Magoulas called it in 2005) has always existed as long as the internet. Around early 2012, big data in smart grid systems initiated collaboration between smart grid integration companies and data analytics start-ups.
As grids became smarter, grid data analytics also developed, using available technologies such as machine learning techniques. Computing techniques like statistics, machine learning (under artificial intelligence), and data analytics are now being applied in various facets, and the power sector is not left out. As we will see, smart grid analytics gives relevant information that helps set the course of upcoming activities for the effective distribution of power.
Three Things You Should Know About The Current Trends In Smart Grid Analytics
For one, the smart grids analytics market in Europe was projected in 2019 to grow at the compounded rate of 11-12.7% by 2025. This growth is based on how advanced grid technologies are embraced on a broader scale. However, I have observed these trends;
1. There is currently rapid growth in investment in smart grids projects and, subsequently, smart grids analytics.
Many countries in the European Union have invested in smart grids projects and are recording successes. Of the projects, up to 59% are demonstration projects, 32% are for deployment, while 9% are research and development projects. A significant highlight is the smart meter roll-out in Italy that takes up to 71% of these projects aforementioned.
Smart meters are installed in all of these projects, and to get relevant information from the data, smart grids analytics have to be employed. These projects result from increased interest and initiatives channelled to the ongoing energy transformation and sustainability goals.
2. Smart grid analytics work with real-time data even with the increased speed and variety of requirements.
This easy adaptation is because they are entirely computerized and are built on the blocks of advanced technologies. Smart grid analytics can now generate information from high-speed data of various forms needed for the grids’ operation and prior knowledge of what to put in as resources.
3. Digital technologies and cloud computing would continue to improve and allow for more data computation.
Digital data, which highest storage used to be terabytes, is now accessible on larger scales like exabytes and zettabytes. Manual methods and previous ways of analyzing this data are becoming redundant. Also, with the inclusion of renewable energy in the conventional grids, the adaptation of intelligent systems is increasing, and the need for grid data analytics will follow this trend.
Challenges Of Smart Grid Analytics
Despite the enormous advantages and improved technologies, there are still a few challenges. Some include:
- Cost implications – the initial costs of setting up smart grids make many grid operators sceptical about using smart grid analytics. For the grid as well, it usually includes the costs of sensors and other components in making it effective. The analytics themselves are part of what makes the smart grid a modern electric system. However, it is worth the investment to foster a low-carbon economy and a greener world.
- Security concerns – the fact that smart grids allow for two-way communication is a concern as the data is prone to cyber-attacks. Despite this, cybersecurity has continued to improve and is developing better solutions using codes and encrypted data.
- Customer demand – the demand needed for effective use of smart grid analytics is higher than what exists now. Not enough grid operators have adopted analytics, and the low-scale usage is not optimal. More large-scale energy supplying and distributing firms need to embrace the new technologies at this time.
I must re-emphasize that smart grid analytics is crucial to improving smart grids’ efficiency, reliability, and sustainability. And Hive Power provides a SaaS platform with intelligent grid analysis and a flexibility management solution called the Flexibility Orchestrator to help the renewable energy industry key players improve their activities and offer services more desirably.
The use of smart grid analytics benefits both the consumers and the suppliers of power because it improves energy management, allows for more efficient power transmission, and lowers the cost of operation and management of energy. In addition, with the use of analytics, demand would match supply more because of improved decision making.
With the recent push to integrate renewable energy into the existing energy infrastructures, it is becoming clear that there is a need to adjust its operation mode. This need is apparent because most renewable energy sources depend on the weather and are not easy to predict or plan with. Moreover, the power generated from such sources as wind energy and solar energy is highly stochastic. This situation calls for the application of advanced technologies for renewable energy forecasting and scheduling.
Renewable energy forecasting helps foresee what changes are expected in the amount of energy that will be generated in the future. This prior knowledge is informative for energy suppliers to plan the input they put into generating systems. Renewable energy scheduling also works side-by-side with forecasting because it is mainly determined by the predictions made by the energy forecasting models.
How renewable energy forecasting and scheduling work
Recent advancements in artificial intelligence have improved the job of weather forecasting (done by meteorologists) through machine learning. As a result, grid operators can leverage machine learning techniques to determine the amount of renewable energy that will be used and purchased by consumers at a particular time.
Machine learning (which is used for renewable energy forecasting) works because a software system learns patterns from recent data and develops an improved analysis for the future. In order to achieve this, a forecasting model is designed to fit a particular situation over several days. In addition, the data collected must be valid, accurate, reliable, consistent, and complete to be effective.
What You Should Know About Renewable Energy Forecasting And Scheduling
Here are five important things about renewable energy forecasting and scheduling you should know;
1. Renewable energy forecasting is built around short-term forecasting
Forecasting can be done with different horizons: short, medium, and long-term. Short-term forecasting involves forecasting from a few minutes to a few days ahead. It is used for day-to-day activities, and this time frame applies to renewable energy prediction.
The lead times in short-term forecasting are such that the changes in weather over a short period can be analyzed and used to predict the data to be used the next time. Renewable energy forecasting and scheduling require updated and recent data as frequently as possible, and short-term forecasting achieves that. The amazing part of it all is that there would be little or no human interruption with the presence of technology. Such way, errors would be significantly minimized.
2. Decentralized computing plays a prominent role in renewable energy scheduling.
Decentralized computing involves the allocation of both software and hardware to various points of duty. It is not like centralized computing, where all activities stem from a particular place. This form of computing (decentralized computing) is necessary for renewable energy scheduling because of the nature of locations in renewable energy generation and consumption.
For example, in an energy community, several houses may produce power at a time, and some utilities need power. The allotment of power to different places can be done effectively with decentralized computing technologies like the blockchain. It is effective because control has to happen independently from various locations when forecasting predictions require an adjustment.
3. Smart grids allow for renewable energy forecasting and scheduling.
Smart grids are electrical distribution points that are not like the conventional grid. The difference is that they contain many operation and control systems, advanced metering systems, intelligent circuit breakers and boards, and most importantly, renewable energy sources fit in well. Their operations are more efficient and can be readily evaluated because of the availability of needed information at the click of the finger. It is in such a system that renewable energy forecasting and scheduling can thrive.
Advanced forecasting models can be introduced and used to plan how the plants would run, whether solar photovoltaics or wind turbines. The ease in integration occurs because the smart grid already has smart IoT devices for thermal sensing, smart meters, phasor management networks, and the likes.
4. Grids with renewable energy attain stability easier with forecasting models in place.
Grid stability is of utmost importance for the sake of the life span of grids. Grid operators cannot consistently have variations in input and output in the grid happen repeatedly. With accurate renewable energy forecasting models, proper preparation and scheduling would be done, and there would be less frequent stability problems. Renewable energy forecasting and scheduling cut back most excesses when it comes to grid management.
5. The weather is a significant factor in renewable energy forecasting and scheduling.
Renewable energy forecasts are usually a combination of accurate weather predictions and the availability of plants and systems. The weather is a great factor, as the weather changes cause significant changes in the renewable power generated. For example, the variable speed of the wind is proportional to the amount of power generated by wind turbines. In the same way, the intensity of sun rays and the positioning of clouds play a big role in the fluctuations when it comes to solar power.
What makes renewable energy forecasting and scheduling interesting is that it studies the highly influencing factors of effective power generation. This kind of study is immediately applied, rather than just being carried out for nothing. It turns out that analyzing the weather, as Meteomatics does, has a vital role in the forecasting done by Hive Power’s Forecaster.
Renewable Energy Forecasting and Scheduling Solution – Hive Power
Hive Power’s Forecaster is one of our Flexibility Operator’s modules that performs short-term forecasting in a very accurate manner. It simply considers various factors involved in renewable energy-based power generation and uses them in forecasting. Its machine learning models make predictions on the amount of energy that would be used and generated in the future, based on previous data. This data is real-time data which is very helpful because it is used as soon as it is delivered.
Renewable energy forecasting and scheduling are essential for the effectiveness of renewable energy systems. With more observations in the needs of a renewable energy system, new technologies keep springing up, and it fosters development. Therefore, it is crucial to embrace these technologies as they come, especially when they are practical and efficient like this (in renewable energy forecasting and scheduling).
The existing energy sector has its systems and organization. The grid controls all forms of power distribution and is more like transferring power from a source to different receiving ends. However, with the inclusion of renewable energy, the need for decentralized energy resources cannot be overemphasized, and the NEMoGrid project achieves this.
Renewable energy sources are diverse, making them of various applications, and they have different capacities. Also, the amount of energy produced always needs proper regulation because of fluctuations. The NEMoGrid project aimed to make market designs that fit in, especially for energy communities. This way, optimal integration results occur with renewables on the grid.
The NEMoGrid project rounded off this past year (2020). Also, earlier this year, we conducted some evaluations. An important strategy used in this application is blockchain technology. There are nine (9) partners that synergized to make this project a success. The partners are the University of Applied Sciences and Arts of Southern Switzerland (SUPSI), Centre for Solar Energy and Hydrogen research – Zentrum für Sonnenenergie- und Wasserstoff-Forschung, Baden-Württemberg (ZSW), the professorship Cognitive and Engineering Psychology at the Chemnitz University of Technology (TU Chemnitz), Sustainable Innovation (SUST), Wüstenrot, Ngenic, Sonnen, Upplands Energi and Hive Power.
Problems Solved by NEMoGrid
Some other market models existed before the concept of decentralized energy resources used in the NEMoGrid. They had their limitations, and those particular issues became solved in the NEMoGrid project. Some of these problems include:
- Lack of flexibility in double auction markets – the double auction market is not flexible enough to regulate market participants. It involves a high number of participants who have to provide their energy forecasts and some other information. The double auction market could also easily get manipulated to the advantage of any market actor.
- The possibility of the iterative price discovery mechanism being prone to collusion – this method is highly dependent on initial declarations and capitalizes on flexibility. It is not a mechanism that can easily fish out such culprits because of the information available from the start.
How NEMoGrid Works – Decentralized Energy Resources
There are various actors in the market mechanism used in the NEMoGrid project. They include prosumers, energy communities, distribution system operators, middle actors (aggregators, balance-responsible parties, and virtual power plants), and legislators who work with energy regulators. In addition, NEMoGrid applies the distributed control theory to overcome the challenges of other market methods.
Some things influenced the choice of markets mechanisms, including:
- non-complex nature
- robust price formation
- clearing in pseudo-real-time, say every 15 minutes
Beyond meeting these, the market formulation that NEMoGrid operates considers decentralized energy resources. Its settings involve two factors – allowance for a group of end-users to control alterable loads and an independent system operator with a defined business model. These two factors make for a spread in control.
The end users can always control the usage of some devices and loads. These loads include heat pumps, electric boilers, and the likes. For being this flexible, the end-users get a reward from the distributed capital gain. The business model exploited by the independent system operator is such that it does the redistribution based on how flexible the end-users are.
Hive Power’s Role in NEMoGRid
Energy communities have a naturally decentralized structure. The structure entails the smart meters that collect, process, and store data of prosumers. Hive Power has a solution based on blockchain technology and uses its solution in the NEMoGrid project (to effectively manage this system).
The use of blockchain still encounters some challenges that limit its adoption. These challenges (such as scalability issues and privacy) are all considered in the application by Hive Power. Furthermore, with the creation of local energy communities on the blockchain, the economic and technical points of view come into play with the Hive Platform community manager module since the goal is to optimize all resources.
Milestones and Current Progress on the NEMoGrid Project
Currently, we have completed the NEMoGrid project, and evaluation is going on. You can access the results of the user research on the NEMoGrid website. The three pilots where we evaluated this project across Europe were in:
- Rolle, Switzerland
- Björklinge, Sweden
- Wüstenrot, Germany
The peer-to-peer scenario used in the blockchain market in these energy communities proved profitable and successful when evaluated.
There are seven (7) work packages involved in the NEMoGrid project managing decentralized energy resources. They include project management (WP0), the definition of future scenarios up to 2025 (WP1), market and tariff management design (WP2), the modelling framework (WP3), social acceptance (WP4), scalability and reliability (WP5), and dissemination and reporting (WP6). At this time, we have completed everything.
Also, the project received funds from the joint programming initiative, ERA-Net Smart Energy Systems focus Initiative (Smart Grids Plus), supported by the European Union Horizon 2020 research program (under grant agreement No. 646039). Also, national funding agencies supported the partners in the NEMoGrid project (Swiss, German and Swedish). These brought it to the point of being a success.
What’s Next For NemoGrid?
Adopting the solutions proffered by NEMoGrid can be extended to more applications with increasing energy communities. However, we need the stage to allow these communities to thrive and apply the flexibility of markets. The control algorithms in the NEMoGrid project, when used, will maximize the potential of these self-consumption communities.
The coming steps in the future would involve more implementation of the solutions discovered and improvements. There would always be room for additional extensions or extensive research. At the same time, the project has already put its propositions forward in a reasonable and proven manner. Any further complications or limitations in the course of usage will be recorded and worked on as part of maintenance strategies.
Vehicle-to-grid (V2G) technology is a means to a greater end for the world of sustainable energy. Even though V2G is not yet prevalent, the structures necessary for communication between grids and electric vehicles have already started growing with advanced technology. It is essential to note that communication protocols that serve as guidelines in their various applications have to be flexible enough to accommodate change constantly.
Communication protocols guide the interactions between two digitally connected entities. In this case, electric vehicles and grids are the entities. Without standards, there is always a gap and disorderliness. Such chaos is not needed in the exchange of data and the facilitation of communication in the application of V2G (Vehicle-to-grid) technology. The IEC 15118 protocol steps in to solve this problem.
V2G technology can only be implemented swiftly and much more if the points of interaction between the two elements, the vehicle, and the grid, recognize each other. You would agree with me that adaptability makes any product or technology, like the advent of electric vehicle usage, more feasible and desirable. The IEC 15118 protocol is one of the other communication protocols but paves the way for a smooth transition in vehicle-grid integration.
The Focus of V2G Communication Protocols
Many concerns come up when it comes to any kind of data exchange. There is a need for the details (like the specifications & unique identity) of a vehicle to be communicated in V2G. Asides from the fact that details may easily be tracked and need a high level of security, the flexibility of the interactions between EVs, charging systems, and grids are highly required for V2G to thrive.
The IEC 15118 started in 2009 for the Vehicle-to-grid Communication Interface to promote autonomous usage. Interestingly, this protocol is still under development, yet it already gives a platform that allows for a broader scope. As V2G communication is needed to be in place for automatic billing and access to the internet, the IEC 15118 protocol gives a form of global compatibility that applies just as well.
IEC 15118 Protocol: What you should know
Of the two main kinds of community protocols (the front-end protocol and the back-end protocol), I would spotlight the IEC 15118 protocol (which is a front-end protocol. That is as a result of its relevance in V2G technology and its application. Also known as the ISO 15118 protocol, it is one of the International Electrotechnical Commission (IEC) standards for electric vehicles (including trucks). It has some interesting sides to it, as I would explain below.
1. More Advanced Communication with IEC 15118
Compared to a similar protocol, like the IEC 61851, the IEC 15118 communication protocol is more advanced. For example, ISO 15118 gives the requirements for charging load management, billing and metering. It thus promotes bi-directional digital communication, which is the basis for V2G communication.
IEC 61851 can only do basic signalling, like indicating readiness for charging and connection status. However, IEC 15118 is applicable for high-level communication, which is an advancement. This places it at the core of EV charging and even V2G interactions. This way, there is better communication and information transfer between the Electric Vehicle and the Electric Vehicle Supply Equipment (EVSE).
2. Versatile Application of IEC 15118 to Wired and Wireless Charging
In its implementation for charging electric vehicles, you can apply IEC 15118 to both wired (AC and DC) and wireless charging. Since V2G applies to various kinds of electric vehicles, this protocol suits it appropriately.
With the current update on part 8 of the IEC 15118 protocol, you would notice an improvement that would allow for wireless connection. Part 8, which is the Physical layer and data link layer requirements for wireless communication, informs the protocol’s versatility.
3. Security via Digital Certification in IEC 1158
The communication between vehicles and grids (via V2G) with the IEC 15118 protocol is more secure. This is a result of the use of digital certificates. In addition, public key infrastructures issue and manage digital certificates. These certificates link people, systems, and keys.
Like passcodes (but more complex), encrypted data is used in IEC 15118 to keep information safe. This way, the limit of insecurities in V2G communication is eliminated. Even digital signatures can be created and used as and when due. If, at any time, for any reason, a digital certificate is no longer trusted, the public key can be reversed. Also, these security features have time limits and make it harder to cheat on the system.
4. Automated Authorization
Using IEC 15118, there is no need to do any other thing at the point of shedding excess power from an electric vehicle to the grid asides from doing the necessary plugging. The automated system allows the system to authenticate the identity of the two sides in communication. It uses different authentication schemes like the Plug and Charge technology, enabling the vehicle to authenticate and identify itself on behalf of the driver.
The use of RFIDs (Radio Frequency Identification) can be aptly applied in the use of IEC 15118 as a means of external identification. Low power radio waves are used in this application to identify the vehicle and automatically carry out authentication.
5. Standard Nature of the IEC 15118 Protocol
ISO/IEC 15118 is a protocol that forms part of the Combined Charging System (CCS) – a group of standards for hardware and software in charging systems. The CCS agrees to use this to enhance charging that can be operated with various specifications.
The International Organization for Standardization (ISO) also recognizes the IEC 15118 protocol for V2G communication. Being an international body made of different national standards organizations that set standards, the ISO is globally recognized.
With Hive Power’s Flexibility Manager Module, anywhere V2G would be implemented, charging and discharging can be coordinated easily. This is done by maximizing devices that can be remotely controlled under this module. The Hive platform also provides a means of improving the accuracy of energy data and enhancing smart grids.
Generally, the interoperability and openness of IEC 15118 make it fit in as a V2G communication protocol well. Yet, it is not at the level it should be in the market. Moreover, due to the nature of the V2G technology as one which is still under development, the entire structure needs to keep improving to aid more advanced communication between the digitally communicating elements.
The introduction of demand-side response meets the preferences of the consumer of energy and helps the energy supply systems to remain balanced. Even though business owners and large-scale commercial corporations were the first to take advantage of this development for the sake of profits, it has moved in its application. Consumer demand-side response is now a point of interest as Demand-side response has its advantages to both a residential consumer and a business owner.
Through demand-side response, the use of power is flexible; as the consumer, you can adjust your energy demand according to your needs. When the United States Energy Independence and Security Act in 2007 defined the term demand response, it described it as all activities related to reducing peak demand through smart pricing and metering, as well as enabling technologies. The whole idea of consumer demand-side response benefits the grid by keeping it stable.
The term Demand-side response was known as Demand-side management (DSM) after the energy crisis in 1979. Various governments wanted to effectively manage demand through different programs because of the issues that arose with energy (fossil fuel then) production. These developments happened both in 1973 and 1979. However, the only thing that is helping Demand-side management thrive now is the availability of communication tools and more technology.
How Consumer Demand-Side Response Works
A distribution grid is responsible for the conveyance of power finally to the end-users. There is a frequency at which power comes into the grid; without renewable energy sources, this frequency is easy to keep stable. You don’t need a high level of control since the power is generated using fossil-based energy sources such as natural gas and coal according to the quantity.
However, including renewable energy sources like solar and wind energy, the input rate is unpredictable. Therefore, the grid operators need the consumers’ cooperation to regulate the power flow to the grid for a reward. Based on requirements and current state, the consumer reduces his power usage and avoids wastage whenever notified.
For a large-scale business or an industrial setting, the demand-side response is very significant because the amount of valuable power that could be wasted is high. Despite their relatively small power capacity, residential consumers can also be participants in demand-side response. With the introduction of advanced technologies, operators can coordinate the demand-side response without much human input. These technologies would account for all little grits of power that accumulate to significant power.
Smart-grid applications provide real-time data to producers and consumers that help them participate in the demand-side response. They aid the effective communication between consumers and producers of electricity on how much is needed and when needed. Consumers can fix their thresholds, then adjust their usage to maximize the prices.
Applicability of Consumer Demand-Side Response
In domestic areas, homes usually have loads that use electric power. They could be:
- Base loads, which are fixed and non-adjustable to meet basic needs such as lighting and the likes.
- Schedulable loads, which are used at some points in time, usually once a day.
- Flexible loads, like water heaters and air conditioning units, are only used when needed.
A consumer can apply the demand-side response to the control of flexible loads in their house. Since they are not used all through the day, they act as virtual batteries. This power gets channelled elsewhere when they are not in use. So, for example, when the weather does not encourage the residents of a house to use the water heating system, they can decline the power supply meant for that purpose.
Technologies Aiding Consumer Demand-Side Response
Certain technologies have been developed and would continue to emerge to achieve the goals of consumer demand-side response. Simply put, they are used for various functions and carry out specific roles to balance the grids.
- Current regulators such as fuses, limiters, and breakers are necessary to moderate the current flowing in or out of a system at a time.
- Distributed intelligent load controllers use artificial intelligence techniques to regulate and manage electricity load in a building.
- Meters – conventional and prepaid meters – are used traditionally to monitor power consumption rate, usage, and units for the sake of payment according to usage.
- Improved metering systems with centralized communication provide two-way communication, inform the consumer of how much power has been used, and help them make decisions. These decisions border around how much power to pay for and use.
The Hive Platform Flexibility Manager Module has an intelligent system used for effective consumer demand-side response. As a result, consumers do not have to be concerned with the activities involved in shifting loads because advanced devices with this technology carry them out.
What the Future Holds for Consumer Demand-Side Response
The advantages businesses get while performing the demand-side responses are more than the disadvantages. Homes can also be a part of this without having to use conventional methods. Smart technologies will continue to get developed and improved till almost all homes become partakers in demand-side response.
The same way advanced metering infrastructures are taking over the metering systems, more people would be able to participate in demand-side response when the available technologies are adopted on a large scale by the grid operators. With advanced grids becoming more used soon, it would aid demand-side response. That way, we can eliminate power outages, and renewable energy would be more appreciated.
Engaging consumers of electricity will only be possible with appropriate communication between them and the suppliers of power. Consumers can make their preferences virtually when necessary or at the initial stages of installation. Also, due to the flexibility introduced in the recent technologies, they can make changes at any point in time.
The Forecaster: using machine learning and weather forecasts to more accurately predict energy consumption and generation.
“It’s hard to make predictions – especially about the future.”- Robert Storm Petersen
Predicting the future might well be hard, but it’s often necessary to run the business. Energy forecasting is of primary importance in day-to-day market operations. Short-term forecasting generally involves forecast horizons that range from a few minutes to a few days ahead. Energy Suppliers and Distribution System Operators benefit from accurate predictions of power demand and generation because they can optimally orchestrate their flexible assets to achieve their business goals.
Hive Platform’s Forecaster module computes short-term stochastic forecasts of aggregated energy consumption and PV generation, using the most advanced machine learning methods available. Stochastic means that we not only predict the “shape” of the load curve over time but also calculate confidence intervals around this curve.
Prediction intervals are essential when it comes to risk management applications, such as predicting the daily or monthly peak of consumption and efficiently activating peak shaving mechanisms.
How does it work?
There is no crystal ball (unfortunately). Instead, the Hive Platform’s Forecaster predicts future consumption and generation by learning what happened in the past and using information available from the future. The Forecaster consists of an ensemble of machine learning models trained on and continuously re-informed by a rich dataset.
The data fed into these models includes the power signal itself, numerical weather predictions (temperature, solar radiation, wind speed and direction, humidity, and heaps of other parameters), seasonality information (the time of the day, the day of the week, the month of the year, etc.), public holidays, school holidays, and custom events (lockdowns, white nights, major events, religious events, etc.).
Weather is king
Weather is by far the most important external factor affecting energy consumption and generation. For this reason, Hive Power decided to conduct an internal research study to review and select the most performant numerical weather prediction provider.
One important criterion we considered was the availability of historical weather forecasts. Many weather providers do not archive and store their predictions. However, historical weather forecasts are of crucial importance to train an energy prediction model. To be robust and reliable, a model should be trained on the same type of data that will be used at inference time.
To understand this point, think of the following. If we trained the model on actual weather observations, the model would learn to trust the weather signals to a certain level. When using this model to predict the future, we would need to replace weather observations with weather predictions, which will not be 100% accurate by definition of prediction. Hence the model will give too much weight to the weather predictions and fool itself into error.
Conversely, if we train the model on weather predictions, the model will learn to trust less the weather parameters and commit less error at inference time. Still sceptical? Try out yourself. Some of our most wary scientists did so and found exactly what was expected.
Weather forecast benchmark
We looked at about a dozen different providers, filtered out those that did not tick our boxes, and ended up with five finalists. We then asked them for a sample of their data to build a benchmark. We requested a year’s worth of hourly predictions of ground-level temperature and solar radiation generated for a single location at around midnight and covering from 24 to 48 hours ahead, which is the typical forecast horizon of our energy prediction models. We compared these predictions with actual local observations and were stunned to discover Meteomatics’ superior performance.
In the figures below, we show some results. In Figure 1, we plotted the distribution of the discrepancy between observed and predicted temperature. In Figure 2, we overlay the five distributions for a more convenient comparison. A similar situation was found for the solar radiation parameters. It was then clear to us that Meteomatics numerical weather predictions were the most accurate and well-calibrated.
Figure 1 – Distribution of the discrepancy between observed and predicted ground temperature for five different weather providers. The vertical dashed lines indicate the mean of each distribution (only Meteomatics’ mean error is centred on zero). The Mean Absolute Error (MAE) is reported on each chart (the lower, the better).
Figure 2 – The same error distributions of Figure 1 overlaid (after estimating their kernel density). Meteomatics’ curve is the narrowest and the only one that is zero-centred.
The beginning of a strategic partnership
“We were looking for a one-stop-shop that could provide us with up-to-date and detailed weather data covering Europe, which needed to be conveniently available via a RESTful API. We took quite some time to evaluate several numerical weather prediction providers. It was not an easy choice because of our demanding list of criteria, but Meteomatics checked all the boxes and exceeded our expectations. We were after a provider that covered the whole world, with a focus on Europe and especially the Alps, which are a challenging region when it comes to high-resolution weather forecasting. It was clear from our own forecast benchmarking exercise that Meteomatics was the most accurate weather forecaster, particularly over the short term. We were pleased to discover the rich plethora of standard and advanced weather parameters that Meteomatics’ API offers. We are thrilled to have switched to Meteomatics as our weather data provider of choice, and we foresee a long and fruitful partnership with them.”, said Gianluca Corbellini, Managing Director at Hive Power.
You can read more about this in Meteomatics’ latest press release: https://www.meteomatics.com/en/meteomatics-and-hive-power-agree-strategic-partnership/ .
Stay tuned for an upcoming webinar (26th of October 2021) where Hive Power and Meteomatics will guide you through an in-depth look at how numerical weather predictions inform energy prediction models.
Meteomatics and Hive Power Agree Strategic Partnership: Bringing machine learning to more accurately predict energy consumption and generation.
Meteomatics AG is a private weather business, making a rich database of weather information (>7 Petabytes) and weather insights available to users across the globe. Meteomatics provides incredible detail and accuracy with weather forecasts downscaled to 90 meters and up to 5 minutes temporal resolution, globally. All through an easy-to-use RESTful API endpoint.
Providing downscaled forecasts at 90-meter resolution and up to 5-minute temporal resolution for anywhere on the planet, were very important reasons why Hive Power chose to switch to Meteomatics.
Hive Power is a smart grid analytics company with a strong focus on creating innovative solutions to improve grid operations for energy suppliers and grid operators through data-driven artificial intelligence. One of their innovations is the Hive Power Forecaster which computes stochastic energy forecasts to simplify how energy retailers and grid operators manage the aggregated energy production and consumption.
“We were looking for a one-stop-shop that could provide us with up to date and detailed weather data covering Europe, which needed to be conveniently available via a RESTful API. Hence why we were very excited about partnering with Meteomatics, and the commercial possibilities Meteomatics’ API could generate for Hive Power. We took quite some time to evaluate several numerical weather prediction providers. It was not an easy choice because of our demanding list of criteria, but Meteomatics checked all the boxes and exceeded our expectations. We were after a provider that covered the whole world, with a focus on Europe and especially the Alps, which are a challenging region when it comes to high-resolution weather forecasting. It was clear from our own forecast benchmarking exercise that Meteomatics was the most accurate weather forecaster, particularly over the short term. We were pleased to discover the rich plethora of standard and advanced weather parameters that Meteomatics’ API offers. Moreover, Meteomatics was one of the few providers that easily allowed the retrieval of archived historical weather forecasts, which are of critical importance to correctly train our machine learning models. We are thrilled to have switched to Meteomatics as our weather data provider of choice, and we foresee a long and fruitful partnership with them.’’, said Gianluca Corbellini Managing Director at Hive Power.
Meteomatics Weather API allows Hive Power to inform and enrich their proprietary energy consumption and production forecasting models. Powered by the latest and most advanced machine learning techniques, Hive Power’s Forecaster computes short-term probabilistic predictions to simplify how energy retailers and grid operators manage the aggregated energy production and consumption. Accurate energy predictions are crucial for effective peak-shaving, performant energy trading, robust grid stability and avoidance of congestions.
Meteomatics unique approach to forecast downscaling allows Meteomatics’ API to resolve the challenges local topography can bring to weather, enabling Meteomatics to achieve a very high degree of forecast accuracy. Plus, the breadth of Meteomatics’ weather database covering all forecast timescales: historical data (from 1979), nowcasts, forecasts, probabilistic and seasonal, allows agricultural businesses to improve operational efficiencies and forecast yields.
Just as the progress of a being shows its life, the growth in a nation shows that it has great abilities. Europe, for one, has shown immense growth in the past three to four years alone. Following act 21 of the RED II (Renewable Energy Directive II), efforts have been made to bring Europe to its target for renewable energy by various bodies.
Interestingly, the statistics of the energy economy in the European Union (EU) show an increase in the production of renewable energy. In 2019, it had the biggest share in primary energy production (36.5%) and so looks productive. As more renewable energy projects come up, more technologies come up to keep up.
More so, to achieve the factors on which sustenance energy thrives (energy security, environmental impact mitigation, and social equity), some ways of keeping sustainable energy effective have come up. One of the most effective is energy communities.
In the same way projects in the US like the Butler solar facility, Comanche solar, and the rest are making progress, Europe is maximizing sustainable energy by doing more energy projects. Energy communities in Europe are increasingly helping citizens contribute to renewable energy and see the effect as closely as possible.
One important thing that characterizes energy communities is citizens’ collective and organized action in producing and using sustainable energy. There are some projects currently going on in Europe whose influence is turning the energy sector around. You can find five top ones here, which we will discuss.
1. The Lugaggia Innovation Community (LIC) Project
The Institute of Systems and Applied Electronics (SUPSI) launched a project in 2019, which is gradually coming to completion in 2021. This project, the Lugaggia Innovation Community, was set up as a self-consumption community. After the municipality of Capriasca installed a solar photovoltaic plant on the roof of a kindergarten, users observed that the rate of consumption of this power was low. To maximize the energy, the LIC connects the kindergarten with ten nearby houses.
The technologies applied in LIC are of utmost importance as they apply key advancements. Two technical solutions provide the backbone of the LIC project – a centralized platform for energy management provided by Optimatik and a decentralized control system by Hive Power’s control module. This second solution introduces blockchain technology for a versatile application. The control that it provides caters to synchronization, payments, sensing, and actuation.
The LIC project is a strong one as it was initially experimental but now promises to be an innovation site. Processes are in place to make it as efficient as possible.
This project is being carried out in Zwevegem, a small town in West Flanders, Belgium. RE/SOURCED stands for Renewable Energy Solutions for Urban communities based on Circular Economy policies and DC backbones. It is focused on maximizing sustainable energy, conversion of heritage, and the circular economy.
There are three structural partners to the RE/SOURCED project, namely:
- the Province West-Flanders, and
With these three partners, the project is led by the intercommunale Leiedal and supported by Urban Innovation Actions of the EU. The project aims at transforming a former power station (established in 1912), Transfo, into an energy community.
Transfo is now a multifunctional site with homes, offices and other structures. It is a 10-hectare site preserved for its heritage and with a lot of significance. The citizens of this community are to benefit from the local power grid that is being developed. The focus is currently on making circularity applicable in renewable energy.
The DC grid for the RE/SOURCED project brings together various renewable energy sources – wind turbines, solar panels, and storage facilities. The idea of a circular economy comes into play in using more efficient materials for the demand of steel, copper, lithium, and the like to be met. The factor of material usage is great in the sustainability of energy systems.
With a focus on energy islands, the COMPILE project started in 2018 and is actively in progress. It is centred around the decarbonization of the energy supply process and community building.
There are up to twelve (12) partners in this project, and they all play roles in putting it together. The project has received funding from the European Union’s horizon 2020 research and innovation. It promises to make use of some tools to achieve its goals.
- COOLkit – this combines the elements of COMPILE’s toolset. All of this is for proper management of the energy community by communicating methods, motives and steps.
- GridRule – this tool is to help actors in the project know how to manage and control a microgrid.
- EVrule – for electric vehicle charging, an electrical charging station by Etrel is explored. This allows for a fair distribution of available power for charging.
- HomeRule – this platform allows users to understand the consumption of power and storage. It is connected with the EVrule and ComPilot.
- ComPilot – this tool is a digital platform. It would provide a stage for virtual social energy communities and work with the other tools.
- Value tool – helps consumers or communities that want to start or join the energy community. The tool provides various business models for these prospective users to explore.
4. The SCCALE 20-30-50 project
Scaling up, according to the project’s name, is a major focus of this project. It kicked off on the 7th of June, 2021, aiming to bring Europe closer to the renewable energy goals. RESCoop coordinates it. There are partners from 5 countries – Energy Cities member cities Leuven (Belgium) and Poreč (Croatia), the energy cooperatives Enercoop, Electra, Energie Samen, ZEZ and Ecopower and TU Delft.
The synergy among the technologies of these partners is on the move to power energy communities around Europe. It is set to create 25 energy communities and 34 community projects.
This project seeks to empower prosumers and create a platform where all can play a good role in the energy market. It is a project that is still under development and plans to be demonstrated in 4 countries on a large scale – Italy, Belgium, Spain and Greece.
WiseGRID integrates ICT systems in the distribution grids of energy to ensure flexibility of the grid systems. A set of technologies will be put in place to ensure smarter grids. The use of enhanced storage systems (batteries and heat accumulators) is a special highlight that WiseGRID will use to store energy from renewable sources.
Also, virtual power plants would be used to manage the controls in this project. Of the 21 partners in this project, several include Ampere Energy, ReScoop, Eco power, and so on.
There are a host of components needed for a smart grid to function at its utmost capacity. In 2008 the Department of Energy (DOE) in America put together a task force of some of the foremost thinkers and shapers of the smart grid sector, and they agreed on a few defining characteristics of a smart grid that would be able to meet the needs it was created for; this is what they came up with:
- Enable active participation by consumers
- Accommodate all generation and storage options
- Enable new products, new services and new markets
- Provide power quality for the wide range of needs in a digital economy
- Optimise asset use and operating efficiency
- Expect and respond to system disturbance in a self-healing manner
- Operate resiliently against physical and cyber-attacks as well as natural disasters
From the list above, we see that a lot of communication and data management is necessary for the workability of smart grids, and one of the solutions to this crucial communication need is the (AMI)-advanced metering infrastructure. AMI is a foundational component that enables smart grid technology to work cohesively.
The advanced metering infrastructure (AMI) is an integrated system made up of smart meters, communication networks and data management systems that allows two-way communication between the utilities provider and customer. This infrastructure is an essential step in the modernisation of grid technologies because it directly includes the customer into the working framework of the smart grid, which increases the added value to the services rendered.
Since AMI is a critical infrastructure of the smart grid, it is also deployed with its unique components:
- Smart meters and data concentrators
- Wide-area communication network (WAN)
- Meter data central (MDC) system
- Meter data management (MDM) system
- Home area network (HAN)
This is where meter management systems, or more concise, meter data management systems, come into play.
What is Meter Data Management System And How it Work?
According to OpenEI, “a meter data management system (MDMS) collects and stores meter data from a head-end system and processes that meter data into information that can be used by other utility applications including billing, customer information systems and outage management systems”.
This system is built on the MDC system, whose primary function includes the validation, estimation, and editing (VEE) of meter data that are later passed on to utility systems, even though disruption of meter data flows may occur.
An MDMS is essential to handling the large amounts of data generated through automated metering or the advanced metering infrastructure. It allows loose coupling between systems.
Several automated meter reading (AMR) systems send their data through their respective head-end servers for the VEE routine to fill gaps in their data, creating clean, integrated and bill-ready data sets. Other utility systems like a data warehouse, outage management, or billing also get their data for their specific purposes from MDMS.
Some AMR/AMI systems that provide meter data to MDMS are gas meters, electric meters and water meters. Compared to conventional grid systems, MDMS enables the consumer/customer to view all their consumption data under one structure, with the ability to manage both analogue and interval data to optimise usage and costs.
The Role Of MDMs
Despite its defining role as a data source, the MDMS plays some other functional roles within the larger IT ecosystem. It can be a traffic director, a data repository, a data framing engine, an infrastructure map and an asset management system.
- Traffic director: in this role, the MDMS can connect back-end applications to specific AMR/AMI systems on a dynamic basis; this makes access to data easy and transparent for users.
- Data repository: in this role, MDMS can serve as an intermediary between the back-end applications that request meter information and specific AMR/AMI systems that collect the data. While MDMS is primarily an online transaction processing system, it can act as an interim data repository.
- Data framing engine: in this role, MDMS can assign interval usage data into specific billing determinants to allow billing of complex rates. This comes in handy when customers are on particular incentives such as time-of-day or peak day pricing rate where the pricing varies exponentially.
- Infrastructure map: in this role, MDMS can save a very detailed virtual map of the electric infrastructure components and their interconnections. These components include meters, transformers, distribution circuits, substations and the like. This map is used as a connectivity model to pass that information like outage alarms to outage management systems and other notifications to their respective systems.
- Asset management system: in this role, the infrastructure map that MDMS already has can be augmented with asset data to be used as an asset management system that can come in handy for small-scale utility companies that may be unable to afford a stand-alone asset management system.
There are numerous roles the MDMS can fit into in the ever-evolving smart grid sector. It is, however, worthy to note that there are a few challenges with its deployment, such as data synchronisation, system integration, scalability, system configuration and time synchronisation, which all have to do with the massive amount of data that runs through the MDM system.
Once the amount of data finds a perfect working synergy within the MDMS, these challenges should be a thing of the past.
The Future of MDMS?
The MDMS is meant to provide effective integration with reduced infrastructure complexity that can easily accommodate any change to its numerous parts without affecting the whole system.
In the global energy market, there is growing consumer demand and the rise of the prosumer, driving an increase in the deployment of smart grids, which need working and sustainable components to meet these demands and boost market growth. Like the Hive Platform, which easily plugs to DSO’s MDMs as a data source for our algorithms and smart grid analytics modules.
Other factors like integration of AMI systems with cloud computing and Internet-of-Things (IoT), extensive research and development will drive the global MDMS market further than anticipated.