Elastic

今天來嘗試使用 Elastic 向量 DB 聽說是有針對資料搜尋特化的一個向量DB，那我們就來試試看吧！！

我是使用 Docker 來搭建 Service 所以以下範例都是以 Docker 來操作。
我這邊是使用 Single Node 來架設DB，但因為 Elastic 是可以有多節點來作分散式備份等功能有興趣可以參考最下面的參考資料。

建立環境

Step1. (Option) Create a new docker network
```
docker network create elastic
```
Step2. docker run elastic
這邊有分有無要在 Elastic 內部使用 LLM 設定因為會有關 Ram 使用的大小
普通建構無須在內部使用 LLM
```
docker run --name elasticsearch --net elastic -p 9200:9200 -it -m 1GB -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:8.16.1
```
需使用機器學習功能，例如使用 ELSER 進行語義搜索需要記憶體超過 1GB 的更大容器。
```
docker run -d --name elasticsearch --net elastic -p 9200:9200 -m 6GB -it -e "xpack.ml.use_auto_machine_memory_percent=true" -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:8.16.1
```
補充：
- 不要啟動 security feature (強制使用https選項) ⇒ “xpack.security.enabled=false”
- 設定 single-node ⇒ “discovery.type=single-node”

Step3. Kibana 的 elastic 使用者密碼和註冊令牌
我們在有啟用 security feature 情況下首次登入 elastic (https://0.0.0.0:9200) 是需要輸入帳號密碼的

docker exec -it elasticsearch /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic

就能看到 elastic 重製好的密碼顯示在上面

  ❯ docker exec -it elasticsearch /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic
  WARNING: Owner of file [/usr/share/elasticsearch/config/users] used to be [root], but now is [elasticsearch]
  WARNING: Owner of file [/usr/share/elasticsearch/config/users_roles] used to be [root], but now is [elasticsearch]
  This tool will reset the password of the [elastic] user to an autogenerated value.
  The password will be printed in the console.
  Please confirm that you would like to continue [y/N]y

  Password for the [elastic] user successfully reset.
  New value: 4If*lQS*fXrh3iq_bvnc

  What's next:
      Try Docker Debug for seamless, persistent debugging tools in any container or image → docker debug elasticsearch
      Learn more at https://docs.docker.com/go/debug-cli/

Step4. docker run kibana (GUI Web) 這邊建立一個官方用來方便管理 Elastic 的 GUI Web Kibana \
```
docker run -d --name kibana --net elastic -p 5601:5601 docker.elastic.co/kibana/kibana:8.16.1
```
之後我們就能到 http://0.0.0.0:5601 看到 Kibana Web 一樣如果有啟用 security feature 情況下首次登入Token

Step5. 取得 Kibana Token

docker exec -it elasticsearch /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana

  ❯ docker exec -it elasticsearch /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
  eyJ2ZXIiOiI4LjE0LjAiLCJhZHIiOlsiMTcyLjE4LjAuMjo5MjAwIl0sImZnciI6ImJjMjI0N2Q3ODNlY2FjYWQ1Zjc0YzdlN2I4ZjM1YTU4YWMyYjBjZGM4NTI3Mzc5MWM0Yjc4OTVmNjQxMWIyNDUiLCJrZXkiOiJMOGFKZHBNQlRra0lBWHVNTk40Zzpqa3RxQ1Q4U1JHaXRlb25YMExjdm9RIn0=

  What's next:
      Try Docker Debug for seamless, persistent debugging tools in any container or image → docker debug elasticsearch
      Learn more at https://docs.docker.com/go/debug-cli/

之後我們就能輸入Token，讓 Kibana 跟 Elastic 連接之後就會看到要輸入認證 Code kibana-verification-code

Step6. 取得認證 Code

> docker exec -it kibana /bin/bash
> kibana@da057b8cc1fa:~$ bin/kibana-verification-code
  Kibana is currently running with legacy OpenSSL providers enabled! For details and instructions on how to disable see https://www.elastic.co/guide/en/kibana/8.16/production.html#openssl-legacy-provider
  Your verification code is:  632 200
  kibana@da057b8cc1fa:~$

手動取消安全安全認證

為了之後我們方便開發時連線不用帶金鑰等等…麻煩的設定

Elastic

❯ docker exec -u root -it elasticsearch /bin/bash
  root@1604ebd3d6d6:/usr/share/elasticsearch# cd config/
  root@1604ebd3d6d6:/usr/share/elasticsearch/config# cat elasticsearch.yml
  cluster.name: "docker-cluster"
  network.host: 0.0.0.0

  #----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
  #
  # The following settings, TLS certificates, and keys have been automatically
  # generated to configure Elasticsearch security features on 29-11-2024 07:00:36
  #
  # --------------------------------------------------------------------------------

  # Enable security features
  xpack.security.enabled: false

  xpack.security.enrollment.enabled: true

  # Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
  xpack.security.http.ssl:
  enabled: false
  keystore.path: certs/http.p12

  # Enable encryption and mutual authentication between cluster nodes
  xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
  #----------------------- END SECURITY AUTO CONFIGURATION -------------------------
  root@1604ebd3d6d6:/usr/share/elasticsearch/config#

補充：
這邊是要把 xpack.security.enabled 跟 xpack.security.http.ssl 修改成 false 即可

Kibana

❯ docker exec -u root -it kibana /bin/bash
root@7bf3cda51529:/usr/share/kibana# cd config/
root@7bf3cda51529:/usr/share/kibana/config# cat kibana.yml
  #
  # ** THIS IS AN AUTO-GENERATED FILE **
  #

  # Default Kibana configuration for docker target
  server.host: "0.0.0.0"
  server.shutdownTimeout: "5s"
  elasticsearch.hosts: [ "http://elasticsearch:9200" ]
  monitoring.ui.container.elasticsearch.enabled: true
  root@7bf3cda51529:/usr/share/kibana/config#

補充： \

elasticsearch.hosts: [ “http://elasticsearch:9200” ] 這邊的 elasticsearch 是依照你的 elastic 的 container 名去修改(這是因為 DNS指向的關係) , 並且注意要是 http 開頭

停用後頁面會顯示 Could not retrieve current user, security plugin is not ready 不用怕照樣能用 disable-security

這樣就建立好一個 ElasticSearch 的服務了！！

Reference

[Elasticsearch] 分散式特性 & 分散式搜尋的機制 - 小信豬的原始部落

Elastic

建立環境

手動取消安全安全認證

Reference

OKH@nd