Add vector data to a collection

Add vector data to a collection require an embedding model

Skypydb support all the Ollama, OpenAI, Sentence Transformers, embeddings models, to use the vector part of our database you will need either download Ollama and install an embedding model or use an OpenAI API key.

By default Skypydb use mxbai-embed-large.

1. Create a collection

   To add data to a collection you will need to create a collection:

   Python

   # Create a client
   client = skypydb.VectorClient(
       embedding_provider="ollama",
       embedding_model_config={
           "model": "mxbai-embed-large",
           "base_url": "http://localhost:11434"
       }
   )
   
   # Create a collection
   collection = client.get_or_create_collection("my-documents")
   

   Use the OpenAI model

   Python

   # Create a client
   client = skypydb.VectorClient(
       embedding_provider="openai",
       embedding_model_config={
           "api_key": "your-openai-api-key",
           "model": "text-embedding-3-small"
       }
   )
   
   # Create a collection
   collection = client.get_or_create_collection("my-documents")
   

   Use the Sentence Transformers model

   Python

   # Create a client
   client = skypydb.VectorClient(
       embedding_provider="sentence-transformers",
       embedding_model_config={
           "model": "all-MiniLM-L6-v2"
       }
   )
   
   # Create a collection
   collection = client.get_or_create_collection("my-documents")
   
2. Add data to a collection

   We can now add data to our collection.

   Python

   # Add documents
   collection.add(
       documents=["This is document1", "This is document2"],
       metadatas=[{"source": "notion"}, {"source": "google-docs"}],
       ids=["doc1", "doc2"]
   )