Transfer learning through abstraction using learning vector quantization

Reinforcement learning (RL) enables an agent to find a solution to a problem by interacting with the environment. However, the learning process always starts from scratch andpossibly takes a long time. Here, knowledge transfer betweentasks is considered. In this paper, we argue that an abstraction c...

Full description

Bibliographic Details
Main Authors:	Ahmad Afif, Mohd Faudzi, Hirotaka, Takano, Junichi, Murata
Format:	Conference or Workshop Item
Language:	English English
Published:	2017
Subjects:	TK Electrical engineering. Electronics Nuclear engineering
Online Access:	http://umpir.ump.edu.my/id/eprint/19465/ http://umpir.ump.edu.my/id/eprint/19465/1/Transfer%20Learning%20through%20Abstraction%20Using%20Learning%20Vector%20Quantization.pdf http://umpir.ump.edu.my/id/eprint/19465/2/Transfer%20Learning%20through%20Abstraction%20Using%20Learning%20Vector%20Quantization%201.pdf

id	ump-19465
recordtype	eprints
spelling	ump-194652018-03-23T08:02:09Z http://umpir.ump.edu.my/id/eprint/19465/ Transfer learning through abstraction using learning vector quantization Ahmad Afif, Mohd Faudzi Hirotaka, Takano Junichi, Murata TK Electrical engineering. Electronics Nuclear engineering Reinforcement learning (RL) enables an agent to find a solution to a problem by interacting with the environment. However, the learning process always starts from scratch andpossibly takes a long time. Here, knowledge transfer betweentasks is considered. In this paper, we argue that an abstraction can improve the transfer learning. Modified learning vector quantization (LVQ) that can manipulate its network weights is proposed to perform an abstraction that is expected to provide a simple representation of the transferred knowledge for human interpretation, an adaptation that is expected to train the agent to adapt to new environments and a precaution that is expected to provide a better prior information. At first, the abstraction is performed by extracting an abstract policy out of a learned policy which is acquired through conventional RL method, Qlearning. The abstract policy then is used in a new task as prior information. Here, the adaptation or policy learning as well as new task’s abstract policy generating are performed using only a single operation. Finally, as a precaution of future tasks, a common abstract policy that extracts the similarities of past tasks’ experiences is introduced. Our simulations show that the representation of acquired abstract policy is interpretable, that the modified LVQ successfully performs policy learning as well as generates abstract policy and that the application of generalized common abstract policy produces better results by more effectively guiding the agent when learning a new task. 2017 Conference or Workshop Item NonPeerReviewed application/pdf en http://umpir.ump.edu.my/id/eprint/19465/1/Transfer%20Learning%20through%20Abstraction%20Using%20Learning%20Vector%20Quantization.pdf application/pdf en http://umpir.ump.edu.my/id/eprint/19465/2/Transfer%20Learning%20through%20Abstraction%20Using%20Learning%20Vector%20Quantization%201.pdf Ahmad Afif, Mohd Faudzi and Hirotaka, Takano and Junichi, Murata (2017) Transfer learning through abstraction using learning vector quantization. In: 4th International Conference On Electrical, Control and Computer Engineering (INECCE2017), 16-17 October 2017 , Hotel Adya, Langkawi. pp. 1-8.. (Unpublished)
repository_type	Digital Repository
institution_category	Local University
institution	Universiti Malaysia Pahang
building	UMP Institutional Repository
collection	Online Access
language	English English
topic	TK Electrical engineering. Electronics Nuclear engineering
spellingShingle	TK Electrical engineering. Electronics Nuclear engineering Ahmad Afif, Mohd Faudzi Hirotaka, Takano Junichi, Murata Transfer learning through abstraction using learning vector quantization
description	Reinforcement learning (RL) enables an agent to find a solution to a problem by interacting with the environment. However, the learning process always starts from scratch andpossibly takes a long time. Here, knowledge transfer betweentasks is considered. In this paper, we argue that an abstraction can improve the transfer learning. Modified learning vector quantization (LVQ) that can manipulate its network weights is proposed to perform an abstraction that is expected to provide a simple representation of the transferred knowledge for human interpretation, an adaptation that is expected to train the agent to adapt to new environments and a precaution that is expected to provide a better prior information. At first, the abstraction is performed by extracting an abstract policy out of a learned policy which is acquired through conventional RL method, Qlearning. The abstract policy then is used in a new task as prior information. Here, the adaptation or policy learning as well as new task’s abstract policy generating are performed using only a single operation. Finally, as a precaution of future tasks, a common abstract policy that extracts the similarities of past tasks’ experiences is introduced. Our simulations show that the representation of acquired abstract policy is interpretable, that the modified LVQ successfully performs policy learning as well as generates abstract policy and that the application of generalized common abstract policy produces better results by more effectively guiding the agent when learning a new task.
format	Conference or Workshop Item
author	Ahmad Afif, Mohd Faudzi Hirotaka, Takano Junichi, Murata
author_facet	Ahmad Afif, Mohd Faudzi Hirotaka, Takano Junichi, Murata
author_sort	Ahmad Afif, Mohd Faudzi
title	Transfer learning through abstraction using learning vector quantization
title_short	Transfer learning through abstraction using learning vector quantization
title_full	Transfer learning through abstraction using learning vector quantization
title_fullStr	Transfer learning through abstraction using learning vector quantization
title_full_unstemmed	Transfer learning through abstraction using learning vector quantization
title_sort	transfer learning through abstraction using learning vector quantization
publishDate	2017
url	http://umpir.ump.edu.my/id/eprint/19465/ http://umpir.ump.edu.my/id/eprint/19465/1/Transfer%20Learning%20through%20Abstraction%20Using%20Learning%20Vector%20Quantization.pdf http://umpir.ump.edu.my/id/eprint/19465/2/Transfer%20Learning%20through%20Abstraction%20Using%20Learning%20Vector%20Quantization%201.pdf
first_indexed	2023-09-18T22:27:47Z
last_indexed	2023-09-18T22:27:47Z
_version_	1777416082397593600

Transfer learning through abstraction using learning vector quantization

Similar Items