Adding docs about embeddings settings + adding the embedding.mode: local in mock profile (#1316)
This commit is contained in:
		
							parent
							
								
									bafdd3baf1
								
							
						
					
					
						commit
						baf29f06fa
					
				|  | @ -33,16 +33,20 @@ Are you running out of memory when ingesting files? | ||||||
| To do not run out of memory, you should ingest your documents without the LLM loaded in your (video) memory. | To do not run out of memory, you should ingest your documents without the LLM loaded in your (video) memory. | ||||||
| To do so, you should change your configuration to set `llm.mode: mock`. | To do so, you should change your configuration to set `llm.mode: mock`. | ||||||
| 
 | 
 | ||||||
| In other words, you should update your `settings.yaml` (or your custom configuration file) to set the | You can also use the existing `PGPT_PROFILES=mock` that will set the following configuration for you: | ||||||
| following **before** ingesting your documents: | 
 | ||||||
| ```yaml | ```yaml | ||||||
| llm: | llm: | ||||||
|   mode: mock |   mode: mock | ||||||
|  | embedding: | ||||||
|  |   mode: local | ||||||
| ``` | ``` | ||||||
| 
 | 
 | ||||||
|  | This configuration allows you to use hardware acceleration for creating embeddings while avoiding loading the full LLM into (video) memory. | ||||||
|  | 
 | ||||||
| Once your documents are ingested, you can set the `llm.mode` value back to `local` (or your previous custom value). | Once your documents are ingested, you can set the `llm.mode` value back to `local` (or your previous custom value). | ||||||
| 
 | 
 | ||||||
| You can also use the existing `PGPT_PROFILES=mock` that will set the `llm.mode` to `mock` for you. | 
 | ||||||
| 
 | 
 | ||||||
| ## Supported file formats | ## Supported file formats | ||||||
| 
 | 
 | ||||||
|  |  | ||||||
|  | @ -1,5 +1,8 @@ | ||||||
| server: | server: | ||||||
|   env_name: ${APP_ENV:mock} |   env_name: ${APP_ENV:mock} | ||||||
| 
 | 
 | ||||||
|  | # This configuration allows you to use GPU for creating embeddings while avoiding loading LLM into vRAM | ||||||
| llm: | llm: | ||||||
|   mode: mock |   mode: mock | ||||||
|  | embedding: | ||||||
|  |   mode: local | ||||||
|  |  | ||||||
		Loading…
	
		Reference in New Issue