public
/
openhab-addons
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[porcupineks] Keyword Spotter, initial contribution (#12028) 

* initial contribution

Signed-off-by: Miguel Álvarez Díez <miguelwork92@gmail.com>
This commit is contained in:
GiviMAD 2022-01-22 14:02:02 +01:00 committed by GitHub
parent f75c04d5d4
commit efa8963d20
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 544 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CODEOWNERS
View File

@ -379,6 +379,7 @@
/bundles/org.openhab.voice.marytts/ @kaikreuzer
/bundles/org.openhab.voice.picotts/ @FlorianSW
/bundles/org.openhab.voice.pollytts/ @hillmanr
/bundles/org.openhab.voice.porcupineks/ @GiviMAD
/bundles/org.openhab.voice.voicerss/ @JochenHiller
/itests/org.openhab.binding.astro.tests/ @gerrieg
/itests/org.openhab.binding.avmfritz.tests/ @cweitkamp

5
bom/openhab-addons/pom.xml
View File

@ -1886,6 +1886,11 @@
<artifactId>org.openhab.voice.pollytts</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.openhab.addons.bundles</groupId>
<artifactId>org.openhab.voice.porcupineks</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.openhab.addons.bundles</groupId>
<artifactId>org.openhab.voice.voicerss</artifactId>

13
bundles/org.openhab.voice.porcupineks/NOTICE Normal file
View File

@ -0,0 +1,13 @@
This content is produced and maintained by the openHAB project.
* Project home: https://www.openhab.org
== Declared Project Licenses
This program and the accompanying materials are made available under the terms
of the Eclipse Public License 2.0 which is available at
https://www.eclipse.org/legal/epl-2.0/.
== Source Code
https://github.com/openhab/openhab-addons

58
bundles/org.openhab.voice.porcupineks/README.md Normal file
View File

@ -0,0 +1,58 @@
# Porcupine Keyword Spotter
This voice service allows you to use the PicoVoice product Porcupine as your keyword spotter in openHAB.
Porcupine provides on-device wake word detection powered by deep learning.
This add-on should work on all the platforms supported by Porcupine, if you encounter a problem you can try to run one of the Porcupine java demos on your machine.
Important: No voice data listened by this service will be uploaded to the Cloud.
The voice data is processed offline, locally on your openHAB server by Porcupine.
Once in a while, access keys are validated to stay active and this requires an Internet connection.
## How to use it
After installing, you will be able to access the addon options through the openHAB configuration page under the 'Other Services' section.
There you will need to provide your PicoVoice Api Key.
After that, you can select Porcupine as your default Keyword Spotter in your 'Voice' settings.
## Magic Word Configuration
The magic word to spot is gathered from your 'Voice' configuration.
The default english keyword models are loaded in the addon (also the english language model) so you can use those without adding anything else.
Note that you can use the pico voice platform to generate your own keyword models.
To use them, you should place the generated file under '\<openHAB userdata\>/porcupine' and configure your magic word to match the file name replacing spaces with '_' and adding the extension '.ppn'.
As an example, the file generated for the keyword "ok openhab" will be named 'ok_openhab.ppn'.
The service will only work if it's able to find the correct ppn for your magic word configuration.
#### Build-in keywords
Remember that they only work with the English language model. (read bellow section)
* alexa
* americano
* blueberry
* bumblebee
* computer
* grapefruits
* grasshopper
* hey google
* hey siri
* jarvis
* ok google
* picovoice
* porcupine
* terminator
## Language support
This service currently supports English, German, French and Spanish.
Only the English model binary is included with the addon and will be used if the one for your configured language is not found under '\<openHAB userdata\>/porcupine'.
To get the language model files, go to the [Porcupine repo](https://github.com/Picovoice/porcupine/tree/v2.0/lib/common).
Note that the keyword model you use should match the language model.

24
bundles/org.openhab.voice.porcupineks/pom.xml Normal file
View File

@ -0,0 +1,24 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.openhab.addons.bundles</groupId>
<artifactId>org.openhab.addons.reactor.bundles</artifactId>
<version>3.3.0-SNAPSHOT</version>
</parent>
<artifactId>org.openhab.voice.porcupineks</artifactId>
<name>openHAB Add-ons :: Bundles :: Voice :: Porcupine Keyword Spotter</name>
<dependencies>
<dependency>
<groupId>ai.picovoice</groupId>
<artifactId>porcupine-java</artifactId>
<version>2.0.2</version>
<scope>compile</scope>
</dependency>
</dependencies>
</project>

9
bundles/org.openhab.voice.porcupineks/src/main/feature/feature.xml Normal file
View File

@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<features name="org.openhab.voice.porcupineks-${project.version}" xmlns="http://karaf.apache.org/xmlns/features/v1.4.0">
<repository>mvn:org.openhab.core.features.karaf/org.openhab.core.features.karaf.openhab-core/${ohc.version}/xml/features</repository>
<feature name="openhab-voice-porcupineks" description="Porcupine Keyword Spotter" version="${project.version}">
<feature>openhab-runtime-base</feature>
<bundle start-level="80">mvn:org.openhab.addons.bundles/org.openhab.voice.porcupineks/${project.version}</bundle>
</feature>
</features>

33
bundles/org.openhab.voice.porcupineks/src/main/java/org/openhab/voice/porcupineks/internal/PorcupineKSConfiguration.java Normal file
View File

@ -0,0 +1,33 @@
/**
* Copyright (c) 2010-2022 Contributors to the openHAB project
*
* See the NOTICE file(s) distributed with this work for additional
* information.
*
* This program and the accompanying materials are made available under the
* terms of the Eclipse Public License 2.0 which is available at
* http://www.eclipse.org/legal/epl-2.0
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.openhab.voice.porcupineks.internal;
import org.eclipse.jdt.annotation.NonNullByDefault;
/**
* The {@link PorcupineKSConfiguration} class contains fields mapping thing configuration parameters.
*
* @author Miguel Álvarez - Initial contribution
*/
@NonNullByDefault
public class PorcupineKSConfiguration {
/**
* Api key to use porcupine
*/
public String apiKey = "";
/**
* A higher sensitivity reduces miss rate at cost of increased false alarm rate
*/
public float sensitivity = 0.5f;
}

44
bundles/org.openhab.voice.porcupineks/src/main/java/org/openhab/voice/porcupineks/internal/PorcupineKSConstants.java Normal file
View File

@ -0,0 +1,44 @@
/**
* Copyright (c) 2010-2022 Contributors to the openHAB project
*
* See the NOTICE file(s) distributed with this work for additional
* information.
*
* This program and the accompanying materials are made available under the
* terms of the Eclipse Public License 2.0 which is available at
* http://www.eclipse.org/legal/epl-2.0
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.openhab.voice.porcupineks.internal;
import org.eclipse.jdt.annotation.NonNullByDefault;
/**
* The {@link PorcupineKSConstants} class defines common constants, which are
* used across the whole binding.
*
* @author Miguel Álvarez - Initial contribution
*/
@NonNullByDefault
public class PorcupineKSConstants {
/**
* Service name
*/
public static final String SERVICE_NAME = "Porcupine Keyword Spotter";
/**
* Service id
*/
public static final String SERVICE_ID = "porcupineks";
/**
* Service category
*/
public static final String SERVICE_CATEGORY = "voice";
/**
* Service pid
*/
public static final String SERVICE_PID = "org.openhab." + SERVICE_CATEGORY + "." + SERVICE_ID;
}

329
bundles/org.openhab.voice.porcupineks/src/main/java/org/openhab/voice/porcupineks/internal/PorcupineKSService.java Normal file
View File

@ -0,0 +1,329 @@
/**
* Copyright (c) 2010-2022 Contributors to the openHAB project
*
* See the NOTICE file(s) distributed with this work for additional
* information.
*
* This program and the accompanying materials are made available under the
* terms of the Eclipse Public License 2.0 which is available at
* http://www.eclipse.org/legal/epl-2.0
*
* SPDX-License-Identifier: EPL-2.0
*/
package org.openhab.voice.porcupineks.internal;
import static org.openhab.voice.porcupineks.internal.PorcupineKSConstants.SERVICE_CATEGORY;
import static org.openhab.voice.porcupineks.internal.PorcupineKSConstants.SERVICE_ID;
import static org.openhab.voice.porcupineks.internal.PorcupineKSConstants.SERVICE_NAME;
import static org.openhab.voice.porcupineks.internal.PorcupineKSConstants.SERVICE_PID;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.Future;
import java.util.concurrent.ScheduledExecutorService;
import java.util.logging.Level;
import org.eclipse.jdt.annotation.NonNullByDefault;
import org.eclipse.jdt.annotation.Nullable;
import org.openhab.core.OpenHAB;
import org.openhab.core.audio.AudioFormat;
import org.openhab.core.audio.AudioStream;
import org.openhab.core.common.ThreadPoolManager;
import org.openhab.core.config.core.ConfigurableService;
import org.openhab.core.config.core.Configuration;
import org.openhab.core.voice.KSErrorEvent;
import org.openhab.core.voice.KSException;
import org.openhab.core.voice.KSListener;
import org.openhab.core.voice.KSService;
import org.openhab.core.voice.KSServiceHandle;
import org.openhab.core.voice.KSpottedEvent;
import org.osgi.framework.BundleContext;
import org.osgi.framework.Constants;
import org.osgi.service.component.ComponentContext;
import org.osgi.service.component.annotations.Activate;
import org.osgi.service.component.annotations.Component;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import ai.picovoice.porcupine.Porcupine;
import ai.picovoice.porcupine.PorcupineException;
/**
* The {@link PorcupineKSService} is a keyword spotting implementation based on porcupine.
*
* @author Miguel Álvarez - Initial contribution
*/
@NonNullByDefault
@Component(configurationPid = SERVICE_PID, property = Constants.SERVICE_PID + "=" + SERVICE_PID)
@ConfigurableService(category = SERVICE_CATEGORY, label = SERVICE_NAME, description_uri = SERVICE_CATEGORY + ":"
+ SERVICE_ID)
public class PorcupineKSService implements KSService {
private static final String PORCUPINE_FOLDER = Path.of(OpenHAB.getUserDataFolder(), "porcupine").toString();
private static final String EXTRACTION_FOLDER = Path.of(OpenHAB.getUserDataFolder(), "porcupine", "extracted")
.toString();
private final Logger logger = LoggerFactory.getLogger(PorcupineKSService.class);
private final ScheduledExecutorService executor = ThreadPoolManager.getScheduledPool("OH-voice-porcupineks");
private PorcupineKSConfiguration config = new PorcupineKSConfiguration();
private boolean loop = false;
private @Nullable BundleContext bundleContext;
static {
Logger logger = LoggerFactory.getLogger(PorcupineKSService.class);
File directory = new File(PORCUPINE_FOLDER);
if (!directory.exists()) {
if (directory.mkdir()) {
logger.info("porcupine dir created {}", PORCUPINE_FOLDER);
}
}
File childDirectory = new File(EXTRACTION_FOLDER);
if (!childDirectory.exists()) {
if (childDirectory.mkdir()) {
logger.info("porcupine extraction file dir created {}", EXTRACTION_FOLDER);
}
}
}
@Activate
protected void activate(ComponentContext componentContext, Map<String, Object> config) {
this.config = new Configuration(config).as(PorcupineKSConfiguration.class);
this.bundleContext = componentContext.getBundleContext();
if (this.config.apiKey.isBlank()) {
logger.warn("Missing pico voice api key to use Porcupine Keyword Spotter");
}
}
private String prepareLib(BundleContext bundleContext, String path) throws IOException {
if (!path.contains("porcupine" + File.separator)) {
// this should never happen
throw new IOException("Path is not pointing to porcupine bundle files " + path);
}
// get a path relative to the porcupine bundle folder
String relativePath;
if (path.startsWith("porcupine" + File.separator)) {
relativePath = path;
} else {
relativePath = path.substring(path.lastIndexOf(File.separator + "porcupine" + File.separator) + 1);
}
File localFile = new File(EXTRACTION_FOLDER,
relativePath.substring(relativePath.lastIndexOf(File.separator) + 1));
if (!localFile.exists()) {
URL porcupineResource = bundleContext.getBundle().getEntry(relativePath);
logger.debug("extracting binary {} from bundle to extraction folder", relativePath);
extractFromBundle(porcupineResource, localFile);
} else {
logger.debug("binary {} already extracted", relativePath);
}
return localFile.toString();
}
private void extractFromBundle(URL resourceUrl, File targetFile) throws IOException {
InputStream in = new BufferedInputStream(resourceUrl.openStream());
OutputStream out = new BufferedOutputStream(new FileOutputStream(targetFile));
byte[] buffer = new byte[1024];
int lengthRead;
while ((lengthRead = in.read(buffer)) > 0) {
out.write(buffer, 0, lengthRead);
out.flush();
}
in.close();
out.close();
}
@Override
public String getId() {
return SERVICE_ID;
}
@Override
public String getLabel(@Nullable Locale locale) {
return SERVICE_NAME;
}
@Override
public Set<Locale> getSupportedLocales() {
return Set.of(Locale.ENGLISH, new Locale("es"), Locale.FRENCH, Locale.GERMAN);
}
@Override
public Set<AudioFormat> getSupportedFormats() {
return Set
.of(new AudioFormat(AudioFormat.CONTAINER_WAVE, AudioFormat.CODEC_PCM_SIGNED, false, 16, null, 16000L));
}
@Override
public KSServiceHandle spot(KSListener ksListener, AudioStream audioStream, Locale locale, String keyword)
throws KSException {
Porcupine porcupine;
if (config.apiKey.isBlank()) {
throw new KSException("Missing pico voice api key");
}
BundleContext bundleContext = this.bundleContext;
if (bundleContext == null) {
throw new KSException("Missing bundle context");
}
try {
porcupine = initPorcupine(bundleContext, locale, keyword);
} catch (PorcupineException | IOException e) {
throw new KSException(e);
}
Future<?> scheduledTask = executor.submit(() -> processInBackground(porcupine, ksListener, audioStream));
return new KSServiceHandle() {
@Override
public void abort() {
logger.debug("stopping service");
loop = false;
try {
Thread.sleep(100);
} catch (InterruptedException e) {
}
scheduledTask.cancel(true);
}
};
}
private Porcupine initPorcupine(BundleContext bundleContext, Locale locale, String keyword)
throws IOException, PorcupineException {
// Suppress library logs
java.util.logging.Logger globalJavaLogger = java.util.logging.Logger
.getLogger(java.util.logging.Logger.GLOBAL_LOGGER_NAME);
Level currentGlobalLogLevel = globalJavaLogger.getLevel();
globalJavaLogger.setLevel(java.util.logging.Level.OFF);
String bundleLibraryPath = Porcupine.LIBRARY_PATH;
if (bundleLibraryPath == null) {
throw new PorcupineException("Unsupported environment, ensure Porcupine is supported by your system");
}
String libraryPath = prepareLib(bundleContext, bundleLibraryPath);
String alternativeModelPath = getAlternativeModelPath(bundleContext, locale);
String modelPath = alternativeModelPath != null ? alternativeModelPath
: prepareLib(bundleContext, Porcupine.MODEL_PATH);
String keywordPath = getKeywordResourcePath(bundleContext, keyword, alternativeModelPath == null);
logger.debug("Porcupine library path: {}", libraryPath);
logger.debug("Porcupine model path: {}", modelPath);
logger.debug("Porcupine keyword path: {}", keywordPath);
logger.debug("Porcupine sensitivity: {}", config.sensitivity);
try {
return new Porcupine(config.apiKey, libraryPath, modelPath, new String[] { keywordPath },
new float[] { config.sensitivity });
} finally {
// restore log level
globalJavaLogger.setLevel(currentGlobalLogLevel);
}
}
private String getPorcupineEnv() {
// get porcupine env from resolved library path
String searchTerm = "lib" + File.separator + "java" + File.separator;
String env = Porcupine.LIBRARY_PATH.substring(Porcupine.LIBRARY_PATH.indexOf(searchTerm) + searchTerm.length());
env = env.substring(0, env.indexOf(File.separator));
return env;
}
private @Nullable String getAlternativeModelPath(BundleContext bundleContext, Locale locale) throws IOException {
String modelPath = null;
if (locale.getLanguage().equals(Locale.GERMAN.getLanguage())) {
Path dePath = Path.of(PORCUPINE_FOLDER, "porcupine_params_de.pv");
if (Files.exists(dePath)) {
modelPath = dePath.toString();
} else {
logger.warn(
"You can provide a specific model for de language in {}, english language model will be used",
PORCUPINE_FOLDER);
}
} else if (locale.getLanguage().equals(Locale.FRENCH.getLanguage())) {
Path frPath = Path.of(PORCUPINE_FOLDER, "porcupine_params_fr.pv");
if (Files.exists(frPath)) {
modelPath = frPath.toString();
} else {
logger.warn(
"You can provide a specific model for fr language in {}, english language model will be used",
PORCUPINE_FOLDER);
}
} else if (locale.getLanguage().equals("es")) {
Path esPath = Path.of(PORCUPINE_FOLDER, "porcupine_params_es.pv");
if (Files.exists(esPath)) {
modelPath = esPath.toString();
} else {
logger.warn(
"You can provide a specific model for es language in {}, english language model will be used",
PORCUPINE_FOLDER);
}
}
return modelPath;
}
private String getKeywordResourcePath(BundleContext bundleContext, String keyWord, boolean allowBuildIn)
throws IOException {
String localKeywordFile = keyWord.toLowerCase().replace(" ", "_") + ".ppn";
Path localKeywordPath = Path.of(PORCUPINE_FOLDER, localKeywordFile);
if (Files.exists(localKeywordPath)) {
return localKeywordPath.toString();
}
if (allowBuildIn) {
try {
Porcupine.BuiltInKeyword.valueOf(keyWord.toUpperCase().replace(" ", "_"));
} catch (IllegalArgumentException e) {
throw new IllegalArgumentException(
"Unable to find model file for configured wake word neither is build-in. Should be at "
+ localKeywordPath);
}
String env = getPorcupineEnv();
String keywordPath = "porcupine/resources/keyword_files/" + env + "/" + keyWord.replace(" ", "_") + "_"
+ env + ".ppn";
return prepareLib(bundleContext, keywordPath);
} else {
throw new IllegalArgumentException(
"Unable to find model file for configured wake word; there are no build-in wake words for your language. Should be at "
+ localKeywordPath);
}
}
private void processInBackground(Porcupine porcupine, KSListener ksListener, AudioStream audioStream) {
int numBytesRead;
// buffers for processing audio
int frameLength = porcupine.getFrameLength();
ByteBuffer captureBuffer = ByteBuffer.allocate(frameLength * 2);
captureBuffer.order(ByteOrder.LITTLE_ENDIAN);
short[] porcupineBuffer = new short[frameLength];
this.loop = true;
while (loop) {
try {
// read a buffer of audio
numBytesRead = audioStream.read(captureBuffer.array(), 0, captureBuffer.capacity());
if (!loop) {
break;
}
// don't pass to porcupine if we don't have a full buffer
if (numBytesRead != frameLength * 2) {
Thread.sleep(100);
continue;
}
// copy into 16-bit buffer
captureBuffer.asShortBuffer().get(porcupineBuffer);
// process with porcupine
int result = porcupine.process(porcupineBuffer);
if (result >= 0) {
logger.debug("keyword detected!");
ksListener.ksEventReceived(new KSpottedEvent());
}
} catch (IOException | PorcupineException | InterruptedException e) {
String errorMessage = e.getMessage();
ksListener.ksEventReceived(new KSErrorEvent(errorMessage != null ? errorMessage : "Unexpected error"));
}
}
porcupine.delete();
logger.debug("Porcupine stopped");
}
}

19
bundles/org.openhab.voice.porcupineks/src/main/resources/OH-INF/config/config.xml Normal file
View File

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<config-description:config-descriptions
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:config-description="https://openhab.org/schemas/config-description/v1.0.0"
xsi:schemaLocation="https://openhab.org/schemas/config-description/v1.0.0
https://openhab.org/schemas/config-description-1.0.0.xsd">
<config-description uri="voice:porcupineks">
<parameter name="apiKey" type="text" required="true">
<label>Pico Voice API Key</label>
<description>API key from PicoVoice, required to use Porcupine.</description>
</parameter>
<parameter name="sensitivity" type="decimal" min="0" max="1">
<label>Sensitivity</label>
<description>Spot sensitivity, a higher sensitivity reduces miss rate at cost of increased false alarm rate.</description>
<default>0.5</default>
</parameter>
</config-description>
</config-description:config-descriptions>

8
bundles/org.openhab.voice.porcupineks/src/main/resources/OH-INF/i18n/porcupineks.properties Normal file
View File

@ -0,0 +1,8 @@
voice.config.porcupineks.apiKey.label = Pico Voice API Key
voice.config.porcupineks.apiKey.description = API key from PicoVoice, required to use Porcupine.
voice.config.porcupineks.sensitivity.label = Sensitivity
voice.config.porcupineks.sensitivity.description = Spot sensitivity, a higher sensitivity reduces miss rate at cost of increased false alarm rate.
# service
service.voice.porcupineks.label = Porcupine Keyword Spotter

1
bundles/pom.xml
View File

@ -397,6 +397,7 @@
<module>org.openhab.voice.marytts</module>
<module>org.openhab.voice.picotts</module>
<module>org.openhab.voice.pollytts</module>
<module>org.openhab.voice.porcupineks</module>
<module>org.openhab.voice.voicerss</module>
</modules>