Semantic search on numeric data

Question

0.00/5 (No votes)

See more:

I have a dataset in csv format. Most columns have numbers in them. The data is mostly number based, with little text.

A sample of my dataset:

Test_name Execution_date User_name Total_tests Passed_tests Failed_tests
test1, 01-02-2020, Marshall, 15, 3, 12
test2, 06-11-2021, Bruce, 6, 5, 1
test3, 08-10-2023, Mathers, 5, 3, 2
test4, 07-06-2023, Three, 2, 1, 1

I want to build a semantic search engine that can answer questions like the following:

Which user has executed the maximum number of tests?
On what date did a particular user execute tests?
Which date observes the maximum number of failed tests?

What I have tried:

I tried Amazon Kendra, Azure Cognitive Search, ChatGPT, Bard, built my own semantic search engine from scratch, but none of them are performing well on numeric data. They're all more text based.

One idea I have is, if I change the numbers to text, will the models be able to predict well?

So for my requiement, what would you suggest me to try? Please shed some light on this.

Posted 24-Nov-23 7:49am

Apoorva666

Add a Solution

Comments

[no name] 24-Nov-23 14:09pm

Simple LINQ / SQL query. How do you sharpen a pencil with a chain saw?

Apoorva666 24-Nov-23 23:44pm

We don't know what exactly the user query is gonna be like, the question can be framed in any way, which is why I'm looking for a tool with semantic search capabilities. For example, "Which user has executed the maximum number of tests" can also be asked as "who ran the maximum number of test operations".

[no name] 25-Nov-23 12:09pm

You're looking for "one size fits all" with an inadequate specification. You also keep asking the same questions ... expecting others to do the research and "learning" for you. And unless you accept that (currently) you need to "bin" your "numeric data", you will keep asking the same pointless questions.

Apoorva666 26-Nov-23 5:30am

Look, I've done lot of research in this regard. I've spent months on this. If you read my question properly, you'll see that I've clearly stated "what I've tried". I'm still a student. I'm not experienced like you guys. I've got no mentors or anyone. My questions may seem silly or repetitive to you, but that is how we newbies who are confused about a lot of things question & eventually learn. And I'm asking for guidance, not the complete code which I can copy-paste, run in an IDE & get the perfect expected output. If you don't understand my question & if you're interested in helping out, do let me know what I need to be more clear about. If you don't have an answer, move on. No need to be so cold.

[no name] 26-Nov-23 15:15pm

"Semantic Search" requires "richly structured source data" (e.g. XML). Yours is not "structured" in that sense.

Apoorva666 27-Nov-23 5:11am

Ok. Thank you.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Maciej Los · Answer 1 · 2023-11-26T05:52:00

I think you don't understand what semantic analysis[^] are.

I'd suggest to start with Stanford's Univerity publication: Learning Word Vectors for Sentiment Analysis[^]. Then, use its Large Movie Review Dataset[^] to create your own model.

In other words - your dataset can not be used for semantic analysis.