{"id":1,"date":"2024-12-02T08:22:13","date_gmt":"2024-12-02T08:22:13","guid":{"rendered":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/?p=1"},"modified":"2025-04-23T12:22:36","modified_gmt":"2025-04-23T11:22:36","slug":"deepqlearningtspproject","status":"publish","type":"post","link":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/2024\/12\/02\/deepqlearningtspproject\/","title":{"rendered":"Deep Q-Learning Research Project"},"content":{"rendered":"\n<p>For my 2024\/2025 research project, I investigated whether Deep Q-Learning (DQL) could overcome the combinatorial explosion associated with the Travelling Salesman Problem. I ran DQL with different parameter tunings, finding that by adjusting the exploration decay, minimum exploration and learning rate could improve performance. I then compared with the Genetic Algorithm (GA) for both its solution cost and runtime. Some of the results are shown below:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"658\" src=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/final_grid_of_graphs-1024x658.png\" alt=\"\" class=\"wp-image-56\" srcset=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/final_grid_of_graphs-1024x658.png 1024w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/final_grid_of_graphs-300x193.png 300w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/final_grid_of_graphs-768x493.png 768w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/final_grid_of_graphs.png 1107w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Performance of DQL with various parameter combinations.<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"659\" src=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/epsilon_decay_0.05_final-1024x659.png\" alt=\"\" class=\"wp-image-57\" srcset=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/epsilon_decay_0.05_final-1024x659.png 1024w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/epsilon_decay_0.05_final-300x193.png 300w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/epsilon_decay_0.05_final-768x494.png 768w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/epsilon_decay_0.05_final.png 1105w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Several parameter combinations run again with a minimum exploration rate of 0.05.<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"562\" height=\"455\" src=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/DQL_vs_GA.png\" alt=\"\" class=\"wp-image-55\" srcset=\"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/DQL_vs_GA.png 562w, https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2024\/12\/DQL_vs_GA-300x243.png 300w\" sizes=\"auto, (max-width: 562px) 100vw, 562px\" \/><figcaption class=\"wp-element-caption\">Cost vs Iterations comparison of DQL and GA performance<\/figcaption><\/figure>\n\n\n\n<p>The results found showed that DQL could indeed be applied to the TSP, but the results were inferior to that of the GA, both with regards to runtime and final cost.<\/p>\n\n\n\n<p>My full report can be viewed <a href=\"http:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-content\/uploads\/sites\/72\/2025\/04\/DQL_for_the_TSP.pdf\">Here<\/a>.<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>For my 2024\/2025 research project, I investigated whether Deep Q-Learning (DQL) could overcome the combinatorial explosion associated with the Travelling [&hellip;]<\/p>\n","protected":false},"author":87,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[5],"tags":[],"class_list":["post-1","post","type-post","status-publish","format-standard","hentry","category-projects_and_research"],"_links":{"self":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/posts\/1","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/users\/87"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/comments?post=1"}],"version-history":[{"count":4,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/posts\/1\/revisions"}],"predecessor-version":[{"id":224,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/posts\/1\/revisions\/224"}],"wp:attachment":[{"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/media?parent=1"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/categories?post=1"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lancaster.ac.uk\/stor-i-student-sites\/mark-holcroft\/wp-json\/wp\/v2\/tags?post=1"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}