Skip to main content
SpringerLink
Log in

The Need for and Application of Data Analytics in Distribution System Studies

  • Chapter
  • First Online:
Data Analytics-Based Demand Profiling and Advanced Demand Side Management for Flexible Operation of Sustainable Power Networks

Part of the book series: Springer Theses ((Springer Theses))

  • 378 Accesses

Abstract

The research reported in this thesis deals with two important and very timely aspects of the future power system operation - assessment of demand flexibility and advanced demand side management (DSM) facilitating flexible and secure operation of the power network. The thesis provides a very clear and comprehensive literature review in these two areas and states precisely the original contributions of the research. It first demonstrates the benefits of data mining for a reliable assessment of demand flexibility and its composition even with very limited observability of the end-users. It then illustrates the importance of accurate load modelling for efficient application of DSM and considers, for the first time, different criteria in designing DSM programme to achieve several objectives of the network performance simultaneously, not only demand reduction as it is the current practice. Finally, it demonstrates the importance of considering realistic assumptions when planning and estimating the success of DSM programs, as these could have a strong impact on the network performance at both transmission and distribution level. The findings of the thesis have both scientific and practical significance, and can be further used as a guidance for application in industry. The thesis also represents an excellent basis for the continuation of research in the aforementioned areas. The research results presented in the thesis have been published in 2 leading international journal papers, 13 international conference papers and embeded in 21 technical reports. Finally, taking all the above into account, including excellent technical presentation of the thesis and a successful defence of the research by the student, the examiners agreed to award the thesis as written with no corrections needed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sebastiani F (2002) Machine learning in automated text categorization. ACM Comput Surv (CSUR) 34:1–47

    Article  Google Scholar 

  2. Managing big data for smart grids and smart meters. IBM Corporation, whitepaper (May 2012), 2012

    Google Scholar 

  3. Chicco G (2015) A multi-faceted view on the characterisation of electrical demand. Invited talk at The University of Manchester

    Google Scholar 

  4. Dong Z, Zhang P (2010) Emerging techniques in power system analysis. Springer

    Google Scholar 

  5. Hong T, Gao DW, Laing T, Kruchten D, Calzada J (2018) Training energy data scientists: universities and industry need to work together to bridge the talent gap. IEEE Power Energy Mag 16:66–73

    Article  Google Scholar 

  6. Weiss SM, Indurkhya N (1998) Predictive data mining: a practical guide. Morgan Kaufmann

    Google Scholar 

  7. Yiteng Z, Yew-Soon O, Tsang IW (2014) The emerging “big dimensionality”. Comput Intell Mag IEEE 9:14–26

    Article  Google Scholar 

  8. Witten IH, Frank E (2005) Data mining: practical machine learning tools and techniques, 2nd edn. Elsevier Science

    Google Scholar 

  9. Han J, Kamber M, Pei J (2011) Data mining: concepts and techniques. Elsevier

    Google Scholar 

  10. Broderick RJ, Williams JR (2013) Clustering methodology for classifying distribution feeders. In: 2013 IEEE 39th photovoltaic specialists conference (PVSC), Tampa, FL, USA, pp 1706–1710

    Google Scholar 

  11. Mori H (2006) State-of-the-art overview on data mining in power systems. In: Power systems conference and exposition, 2006. PSCE ‘06. 2006 IEEE PES, Atlanta, GA, USA, pp 33–34

    Google Scholar 

  12. Matlab I., The MathWorks, statistics and machine learning toolbox user’s guide R2015b, 2015

    Google Scholar 

  13. Smyth GK (2002) Nonlinear regression. Encyclopedia of environmetrics

    Google Scholar 

  14. Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv (CSUR) 31:264–323

    Article  Google Scholar 

  15. Zhao H (2013) Decision tree technology in data classification. In: Applied mechanics and materials, vol 268, pp 1752–1757

    Article  Google Scholar 

  16. Kamalasadan S (1998) Application of artificial intelligence techniques in power systems. Asian Institute of Technology, Bangkok

    Google Scholar 

  17. Haykin SS (2009) Neural networks and learning machines/Simon Haykin. Prentice Hall, New York

    Google Scholar 

  18. Beale MH, Hagan MT, Demuth HB (2017) Matlab neural network toolbox—user’s guide

    Google Scholar 

  19. Nawi NM, Atomi WH, Rehman M (2013) The effect of data pre-processing on optimized training of artificial neural networks. Procedia Technol 11:32–39

    Article  Google Scholar 

  20. Wang T, Zhang G, Zhao J, He Z, Wang J, Pérez-Jiménez MJ (2015) Fault diagnosis of electric power systems based on fuzzy reasoning spiking neural P systems. IEEE Trans Power Syst 30:1182–1194

    Article  Google Scholar 

  21. Urquhart AJ, Thomson M (2015) Impacts of demand data time resolution on estimates of distribution system energy losses. IEEE Trans Power Syst 30:1483–1491

    Article  Google Scholar 

  22. Oracle Database Online Documentation. [Available].Online: http://docs.oracle.com/cd/B28359_01/datamine.111/b28129/regress.htm#CIHHFFHB

  23. Gerbec D, Gasperic S, Gubina F (2003) Determination and allocation of typical load profiles to the eligible consumers. In: 2003 IEEE Bologna Power Tech Conference Proceedings, Bologna, Italy, vol 1, 5 p

    Google Scholar 

  24. Liu G, Yu Y, Gao F, Zhu W (2015) Research of smart distribution network big data model. In: 23rd international conference on electricity distribution, CIRED, Lyon, France, pp 15–18

    Google Scholar 

  25. De Comité F, Gilleron R, Tommasi M (2003) Learning multi-label alternating decision trees from texts and data. In: Machine learning and data mining in pattern recognition. Springer, pp 35–49

    Google Scholar 

  26. Chicco G, Napoli R, Piglione F (2006) Comparisons among clustering techniques for electricity customer classification. IEEE Trans Power Syst 21:933–940

    Article  Google Scholar 

  27. Chenchouni H, Menasria T, Neffar S, Chafaa S, Bradai L, Chaibi R, Mekahlia MN, Bendjoudi D, Bachir AS (2015) Spatiotemporal diversity, structure and trophic guilds of insect assemblages in a semi-arid Sabkha ecosystem. Peer J 3:e860

    Article  Google Scholar 

  28. Srinivasan D, Ng WS, Liew AC (2006) Neural-network-based signature recognition for harmonic source identification. IEEE Trans Power Delivery 21:398–405

    Article  Google Scholar 

  29. Yang S-L, Shen C (2013) A review of electric load classification in smart grid environment. Renew Sustain Energy Rev 24:103–110

    Article  Google Scholar 

  30. Craven M, McCallum A, PiPasquo D, Mitchell T, Freitag D (1998) Learning to extract symbolic knowledge from the World Wide Web. DTIC Document

    Google Scholar 

  31. Hearst MA (1999) Untangling text data mining. Presented at the Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, College Park, Maryland

    Google Scholar 

  32. Bunke H (2003) Graph-based tools for data mining and machine learning. In: Machine learning and data mining in pattern recognition. Springer, pp 7–19

    Google Scholar 

  33. Schapire RE, Singer Y (2000) BoosTexter: a boosting-based system for text categorization. Mach Learn 39:135–168

    Article  Google Scholar 

  34. Spasić I, Greenwood M, Preece A, Francis N, Elwyn G (2013) FlexiTerm: a flexible term recognition method. J Biomed Semant 4:1–15

    Article  Google Scholar 

  35. Manning CD, Raghavan P, Schütze H (2008) Scoring, term weighting and the vector space mode. In: Introduction to information retrieval, vol 100, pp 2–4

    Google Scholar 

  36. Singhal AK (1997) Term weighting revisited. Cornell University

    Google Scholar 

  37. Seki Y (2002) Sentence extraction by tf/idf and position weighting from newspaper articles. In: Proceedings of the Third NTCIR workshop on research in information retrieval, automatic text summarization and question answering (September 2001–October 2002)

    Google Scholar 

  38. Demand side response in the domestic sector—a literature review of major trials. Department of Energy and Climate Change, London

    Google Scholar 

  39. Srivastava A, Cyber infrastructure for the smart grid [Online]. Available: https://sgdril.eecs.wsu.edu/files/files/Lesson%201_2_Motivation%20for%20the%20Smart%20Grid.pdf

  40. Consolidated view on the ETP SG (European technology platform on smart grids) on research, development and demonstration needs in the Horizon 2020 work programme 2016–2017, 2015

    Google Scholar 

  41. Guidelines for power quality monitoring—measurement locations, processing and presentation of data. CIGRE/CIRED C4.112, 2014

    Google Scholar 

  42. Noske S, Falkowski D, Kolodziejczyk K, Helt P (2015) Increase in power network observability as a data source to improve the efficiency of the power network—results of the pilot smart grid project, 15–18 June 2015

    Google Scholar 

  43. Sforna M (2000) Data mining in a power company customer database. Electr Power Syst Res 55:201–209

    Article  Google Scholar 

  44. Alber J (2006) State estimation in PowerFactory: algorithmic aspects. In: Proceedings of RTE-VT workshop, pp 1–9

    Google Scholar 

  45. Kirschen DS, Rosso A, Juan M, Ochoa LF (2012) Flexibility from the demand side. In: 2012 IEEE power and energy society general meeting, San Diego, CA, USA, pp 1–6

    Google Scholar 

  46. Domestic Survey Report, Customer-Led Network Revolution. Northern Powergrid (Northeast) Limited, Northern Powergrid (Yorkshire) Plc, British Gas Trading Limited, University of Durham, [Available]. Online: http://www.networkrevolution.co.uk/resources/project-library, 2014

  47. Hirst E (2002) Barriers to price-responsive demand in wholesale electricity markets. Edison Electric Institute Research Paper

    Google Scholar 

  48. CEER status Review on European regulatory approaches enabling smart grids solutions (“smart regulation”). Council of European Energy Regulators 2014, [Online]. Available: https://www.ceer.eu/documents/104400/-/-/f83fc0d2-bff9-600b-3e0f-14eccad7a8d8

  49. [Online]. Available: http://www.eon.com/en/business-areas/distribution/technology-of-the-future/smart-meters/key-technology-smart-metering.html

  50. Lees M (2014) Enhanced network monitoring report, customer-led network revolution. Northern Powergrid (Northeast) Limited, Northern Powergrid (Yorkshire) Plc, British Gas Trading Limited, University of Durham and EA Technology Ltd., [Online]. Available: http://www.networkrevolution.co.uk/resources/project-library

  51. Frame DF, Ault GW, Huang S (2012) The uncertainties of probabilistic LV network analysis. In: 2012 IEEE power and energy society general meeting, San Diego, CA, USA, pp 1–8

    Google Scholar 

  52. Hasan KN, Wang M, Milanović JV (2018) A survey on demand side management potential in south-east Europe to support transmission network flexibility. In: 2018 IEEE PES innovative smart grid technologies conference Europe (ISGT-Europe), Sarajevo, Bosnia and Herzegovina, pp 1–6

    Google Scholar 

  53. Smart Energy GB, [Online]. Available: https://www.smartenergygb.org/en/faqs

  54. Smart meter data—a guide to your rights and choices, [Online]. Available: http://www.energy-uk.org.uk/policy/smart-meters.html

  55. Jenkins N, Long C, Wu J (2015) An overview of the smart grid in Great Britain. Engineering 1:413–421

    Article  Google Scholar 

  56. The Smart Meter Report: Forecasts, regional breakdowns, costs, and savings for a top IoT device, [Available]. Online: https://www.businessinsider.com/the-smart-meter-report-a-look-at-how-the-smart-meter-market-is-evolving-2015-3?r=US&IR=T

  57. Smart metering equipment technical specifications, [Online]. Available https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/299395/smets.pdf

  58. [Online]. Available: http://www.rexelenergysolutions.co.uk/media/pdfs/EL09_AS230_Single_Phase_Smart_Meter_Technical.pdf

  59. [Online]. Available: http://www.smsmetering.co.uk/wp-content/uploads/2013/08/ME372-Data-Sheet_SMS.pdf

  60. [Online]. Available: http://www.jwsmartmeters.co.uk/Landis-Gyr-E470

  61. Uribe-Pérez N, Hernández L, de la Vega D, Angulo I (2016) State of the art and trends review of smart metering in electricity grids. Appl Sci 6:68

    Article  Google Scholar 

  62. Cost-benefit analyses & state of play of smart metering deployment in the EU-27. European Commission, 2014

    Google Scholar 

  63. Thomas L, Jenkins N (2012) Smart metering for the UK. HubNet Position Paper Series

    Google Scholar 

  64. Samarakoon K, Ekanayake J, Jenkins N (2013) Reporting available demand response. IEEE Trans Smart Grid 4:1842–1851

    Article  Google Scholar 

  65. Introduction to the UK smart grid Sector, [Online]. Available: https://www.techuk.org/

  66. Al-Wakeel A, Wu J, Jenkins N (2017) K-means based load estimation of domestic smart meter measurements. Appl Energy 194:333–342

    Article  Google Scholar 

  67. Xu Y (2015) Probabilistic estimation and prediction of the dynamic response of the demand at bulk supply points. Ph.D. thesis, School of Electrical and Electronic Engineering, University of Manchester

    Google Scholar 

  68. Zhao C, Chenghong GU, Li F (2015) Classification of Low Voltage Distribution Networks Based on Fixed Data. In: 23rd international conference on electricity distribution (CIRED), Lyon, France

    Google Scholar 

  69. Hatziargyriou N, Dimeas A, Korres N, Dova F, Gkavogianni A, Vlachos Y, Koukoula D, Tsitsimelis A (2015) A data repository for automated evaluation of smart grid solutions. In: 23rd International conference on electricity distribution (CIRED), Lyon, France

    Google Scholar 

  70. Rigoni V, Ochoa LF, Chicco F, Navarro-Espinosa A, Gozel T (2015) Representative residential LV feeders: a case study for the north west of England. IEEE Trans Power Syst PP:1–13

    Google Scholar 

  71. Yu N, Shah S, Johnson R, Sherick R, Hong M, Loparo K (2015) Big data analytics in power distribution systems. In: 2015 IEEE power & energy society innovative smart grid technologies conference (ISGT) 2015, pp 1–5

    Google Scholar 

  72. Dong H, Jin M, He R-M, Zhao-yang D (2009) A real application of measurement-based load modeling in large-scale power grids and its validation. IEEE Trans Power Syst 24:1756–1764

    Article  Google Scholar 

  73. Tso S, Lin J, Ho H, Mak C, Yung K, Ho Y (2004) Data mining for detection of sensitive buses and influential buses in a power system subjected to disturbances. IEEE Trans Power Syst 19:563–568

    Article  Google Scholar 

  74. Gahrooei YR, Khodabakhshian A, Hooshmand R-A (2018) A new pseudo load profile determination approach in low voltage distribution networks. IEEE Trans Power Syst 33:463–472

    Article  Google Scholar 

  75. Moraru A, Mladenić D (2013) Complex event processing and data mining for smart cities. In: Conference on data mining and data warehouses (SkiDD 2013), held at the 15th international multiconference on information society (IS-2012), 8th October, Ljubljana, Slovenia

    Google Scholar 

  76. Guo T, Papadopoulos P, Mohammed P, Milanovic JV (2015) Comparison of ensemble decision tree methods for on-line identification of power system dynamic signature considering availability of PMU measurements. In: PowerTech, 2015 IEEE Eindhoven, pp 1–6

    Google Scholar 

  77. Capozzoli A, Piscitelli MS, Brandi S, Grassi D, Chicco G (2018) Automated load pattern learning and anomaly detection for enhancing energy management in smart buildings. Energy 157:336–352

    Article  Google Scholar 

  78. Park DC, El-Sharkawi MA, Marks RJ II, Atlas LE, Damborg MJ (1991) Electric load forecasting using an artificial neural network. IEEE Trans Power Syst 6:442–449

    Article  Google Scholar 

  79. Panapakidis IP, Papagiannis GK, Christoforidis GC (2014) Bus load forecasting via a combination of machine learning algorithms. In: Power engineering conference (UPEC), 2014 49th international universities, Cluj-Napoca, Romania, pp 1–6

    Google Scholar 

  80. Verdú SV, Garcia MO, Senabre C, Marín AG, Franco FG (2006) Classification, filtering, and identification of electrical customer load patterns through the use of self-organizing maps. IEEE Trans Power Syst 21:1672–1682

    Article  Google Scholar 

  81. Rodrigues F, Duarte J, Figueiredo V, Vale Z, Cordeiro M (2003) A comparative analysis of clustering algorithms applied to load profiling. In: Machine learning and data mining in pattern recognition. Springer, pp 73–85

    Google Scholar 

  82. Shi H, Xu M, Li R (2018) Deep learning for household load forecasting—a novel pooling deep RNN. IEEE Trans Smart Grid 9:5271–5280

    Article  Google Scholar 

  83. Liao H, Milanović JV, Rodrigues M, Shenfield A (2018) Voltage sag estimation in sparsely monitored power systems based on deep learning and system area mapping. IEEE Trans Power Delivery 33:3162–3172

    Article  Google Scholar 

  84. Quality of supply and market regulation; survey within Europe, KEMA Consulting, 2006

    Google Scholar 

  85. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. ACM SIGKDD Explorations Newsl 11:10–18

    Article  Google Scholar 

  86. Preece R, Woolley NC, Milanovic JV (2012) NaFIRS data analysis—correlation between quality of supply and network performance data. Internal report, School of Electrical and electronic Engineering, The University of Manchester

    Google Scholar 

  87. The Value of Lost Load (VoLL) for Electricity in Great Britain, final report for OFGEM and DECC. London Economics 2013, [Online]. Available: https://www.ofgem.gov.uk/ofgem-publications/82293/london-economics-value-lost-load-electricity-gbpdf

  88. VirginiaTech Research Data, [Online]. Available: http://www.ari.vt.edu/research-data/

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, The University of Manchester, Manchester, UK

    Dr. Jelena Ponoćko

Authors
  1. Dr. Jelena Ponoćko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jelena Ponoćko .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Ponoćko, J. (2020). The Need for and Application of Data Analytics in Distribution System Studies. In: Data Analytics-Based Demand Profiling and Advanced Demand Side Management for Flexible Operation of Sustainable Power Networks. Springer Theses. Springer, Cham. https://doi.org/10.1007/978-3-030-39943-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-39943-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-39942-9

  • Online ISBN: 978-3-030-39943-6

  • eBook Packages: EnergyEnergy (R0)

Publish with us

Policies and ethics

Search

Navigation