How to Retrieve Vertex IDs While Adding Vertices in a Transaction with JanusGraph.Net? #4407
-
|
I'm currently working on a project involving a graph database (JanusGraph), using the JanusGraph.Net library for operations. I'm facing an issue regarding transactions and vertex IDs retrieval. Here's a snippet of my code: var traversalSource = new GraphTraversalSource().WithRemote(connection);
var vertexTasks = new Dictionary<long, GraphTraversal<Vertex, Vertex>>();
var tx = traversalSource.Tx();
try
{
foreach (var item in DATA)
{
var addVertexTraversal = traversalSource.AddV("DATA").Property(Cardinality.Single, "ID", item.Key);
vertexTasks.Add(item_AidAnalyze.Key, addVertexTraversal);
}
tx.CommitAsync();
Console.WriteLine("Vertex added successfully.");
}
catch (Exception ex)
{
Console.WriteLine($"Failed to add vertex: {ex.Message}");
tx.RollbackAsync();
}In this code, I'm adding vertices to the JanusGraph graph database within a transaction (tx). However, I also need to retrieve the IDs of these vertices for later use in adding properties and edges. Is there a way to achieve this within the context of a transaction? Without using transactions, I achieved my requirements with the following code: Vertex vertex_1 = g.AddV("DATA").Property(Cardinality.Single, "ID", item.Key).Next();
int id = vertex_1.id;But now I can only obtain |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 2 replies
-
|
You don't need to explicitly handle transactions here. You can simply construct a single traversal that creates your vertices and returns the IDs of the created vertices. JanusGraph Server will automatically execute such a traversal in its own transaction with automatic commit and rollback in case an exception occurs. This looks something like this in C#: var t = g.Inject(1); // Inject() is only necessary to get a `GraphTraversal`
foreach (var item in DATA)
{
t.AddV("DATA").Property(Cardinality.Single, "ID", item).Id().Aggregate("ids");
}
var ids = await t.Cap<List<object>>("ids").Promise(tr => tr.Next());I used The important parts here are the |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for your assistance. I'm not yet proficient in Gremlin, and while I understand that consolidating complex requirements into a single Gremlin statement could be beneficial, I encountered some issues when running your code. Therefore, for now, I'll stick with the code I mentioned earlier as it's fully functional and performs well in terms of both performance and functionality. When I ran your code, I encountered the following error: I modified gremlin-server.yaml to set maxContentLength to 1073741824 or 6553600, then obtained the IDs using the following method. int batchSize = 1000; // Set the batch size
List<object> allIds = new List<object>(); // Store all ids in the list
while (true)
{
// Get the next batch of ids
var nextBatch = t.Cap<List<object>>("ids").Next(batchSize);
// If the batch is empty, it means all ids have been retrieved, exit the loop
if (nextBatch.Count() == 0)
{
break;
}
// Add the ids of the current batch to the total list
allIds.AddRange(nextBatch);
}However, when executing var nextBatch = t.Cap<List>("ids").Next(batchSize);, I encountered the following issue.(Exporting IDs can indeed exhaust the JVM's memory, which is hard to understand.) If I modify your code as follows, it will result in poor performance (taking 0.5s to add a single vertex). var t = g.Inject(1); // Inject() is only necessary to get a `GraphTraversal`
foreach (var item in DATA)
{
t.AddV("DATA").Property(Cardinality.Single, "ID", item).Next;
}Therefore, currently, based on the code I provided, I export all vertices from the entire graph to a file using the following (I'm using transactions in this way to avoid the poor performance of "adding a single vertex takes 0.5 seconds".) var traversalSource = new GraphTraversalSource().WithRemote(connection);
var vertexTasks = new Dictionary<long, GraphTraversal<Vertex, Vertex>>();
var tx = traversalSource.Tx();
try
{
foreach (var item in DATA)
{
var addVertexTraversal = traversalSource.AddV("DATA").Property(Cardinality.Single, "ID", item.Key);
vertexTasks.Add(item_AidAnalyze.Key, addVertexTraversal);
}
tx.CommitAsync();
Console.WriteLine("Vertex added successfully.");
}
catch (Exception ex)
{
Console.WriteLine($"Failed to add vertex: {ex.Message}");
tx.RollbackAsync();
}
var traversal = g.V(); // Use the V() step to retrieve all vertices
var vertices = traversal.ToList();
foreach (var item in vertices)
{
//...
}The above code, although not elegant and unable to track the progress of vertex addition, is functional and performs well (adding seventy million vertices with one property each in a few minutes). Thank you once again for your assistance. If in the future I find myself needing to program in your more rigorous manner, I'll give it another try. |
Beta Was this translation helpful? Give feedback.
-
|
In general it seems to me like your batch sizes are too big. I would try whether smaller batches lead to better results, like with 100 or 1000 vertices to be added in a single batch. In your second code listing you have a var nextBatch = t.Cap<List<object>>("ids").Next(batchSize);If I'm not completely mistaken (I haven't checked this), then this will execute the full traversal that you have stored in But again, I think your main problem is that you try to construct way too many vertices in a single batch.
You've added a Your last code listing confuses me a lot. Your are starting a transaction there via If you really want to handle transactions yourself in your code, then it should look more like it's shown in the TinkerPop docs for Gremlin.Net: var tx = g.Tx(); // create a transaction
// spawn a new GraphTraversalSource binding all traversals established from it to tx
var gtx = tx.Begin();
gtx.AddV().Iterate();
gtx.AddV().Iterate();
await tx.CommitAsync();The crucial part here that is missing in your listing is the creation of the new Also, you called the |
Beta Was this translation helpful? Give feedback.
You don't need to explicitly handle transactions here. You can simply construct a single traversal that creates your vertices and returns the IDs of the created vertices. JanusGraph Server will automatically execute such a traversal in its own transaction with automatic commit and rollback in case an exception occurs.
This looks something like this in C#:
I used
asynchere to execute the traversal, but you can of course also do it sy…